Release Notes for SDP - Helix Server Deployment Package Helix Server Components Introduction This document notes changes to the SDP. There are two flavors of the SDP, Unix and Windows. At a high level, the core function of the SDP is similar on Unix and Windows: To maintain an offline copy of the Helix databases, and perform routine offline checkpoints in preparation for backups. At a detailed level, there are differences in both implementation and behavior on the various platforms. Enhancements are needs-driven, and not necessarily applied across Unix and Windows platforms. -------------------------------------------------------------------------- Upgrading the SDP on Unix To incorporate a new version of the SDP into an existing SDP installation, first stage the deployment as in this example, assuming you have put the SDP distribution tar file in /depotdata. Working as user root on a Perforce server: cd /depotdata # If an sdp folder exists here, move it aside, e.g. to # sdp.MOVED_2014-09-11 tar -xzpf sdp.Unix.2014.2.10649.tgz chown -R perforce:perforce sdp.Unix.2014.2.10649.tgz /depotdata/sdp cd /depotdata/sdp/Server/Unix/p4/common/bin # Copy 'p4', 'p4d', 'p4broker', 'p4web', 'p4p', etc. (only # those currently used) into this 'bin' folder. cd /depotdata/sdp/Server/Unix/setup vi mkdirs.sh # Review and adjustments to mkdirs.sh suit your environment. ./mkdirs.sh 1 -test Using 'mkdirs.sh' script with the '-test' flag as illustrated above will deploy to the SDP to /tmp/p4 rather than /p4 area. If working on the Production server directly, DO NOT forget to use the '-test' flag. -------------------------------------------------------------------------- Upgrading the SDP on Windows: Install the SDP onto a different machine - configure sdp_master_config.ini - run create_env.py Manually copy resulting files to the appropriate places in your existing installation. -------------------------------------------------------------------------- Versioning the SDP: We strongly recommend that files in the following locations are versioned in your local Helix server: On Unix: /p4/1/bin /p4/common On Windows: C:\p4\1\bin C:\p4\common C:\p4\config COMING SOON: Helix DVCS and Streams features enable a new method for updating the SDP long-term, greatly simplifying the process of bringing in updates from The Workshop into your environment, and then merging into your local as-deployed SDP (as configured and possibly customized). A simple 'p4 fetch' and 'p4 merge' process. The setup and mechancis will be documented in Q1 2016. -------------------------------------------------------------------------- CHANGE HISTORY: Released: Rev. SDP/MultiArch/2016.2/20972 (2016/11/01) Jobs Fixed since change 20856 up to 20970 (F=Feature, B=Bug): job000616 (F): Add -recent flag to p4verify.sh to verify recent changes. job000626 (B): p4login.*.log files are not being cleaned up by the standard clean-up mechanism (remove_old_logs in backup_function.sh). job000627 (F): Allow for multiple depot volumes. job000628 (B): Fixed missing LOGFILE varialbe issue in p4verify.sh. job000629 (B): Fix formatting of logs to use simple HTML line breaks for Windows SDP. job000630 (B): Various test suite improvements. job000631 (B): fixed issue with broken def'n of P4PORT with multiple SSL prefixes. job000632 (F): Enhanced two-factor auth mechanims. job000633 (F): Tweak to slightly reduce mkdirs.sh configuration. Released: Rev. SDP/MultiArch/2016.2/20856 (2016/10/04) Jobs Fixed since change 20805 up to 20856 (F=Feature, B=Bug): job000618 (F): Enhanced logic to favor init scripts in /p4/N/bin over /etc/init.d. job000619 (B): Fixed quoting bug in p4login. job000620 (F): hms checks for cfg files in /p4/common/config. job000621 (B): Fixed hms bug re: wrong master host discovered in failback testing Released: Rev. SDP/MultiArch/2016.2/20805 (2016/10/02) Jobs Fixed since change 20790 up to 20805 (F=Feature, B=Bug): job000611 (B): Fixed ssh hang issues with hms scripts. job000612 (F): Implemented global/central start stop. job000613 (F): mkrep.sh: Added more replica types and journalcopy support. job000615 (F): Implement checkauth feature in HMS broker_wrapper. Released: Rev. SDP/MultiArch/2016.2/20790 (2016/09/30) Jobs Fixed since change 20755 up to 20790 (F=Feature, B=Bug): job000606 (B): Minor p4 login fix to show P4TICKETS output reliably. job000608 (B): Fix mkrep.sh to use $P4BIN instead of p4. job000609 (F): Added basic p4login-super.sh script. job000610 (B): Added hms dir to mkdirs.sh. Released: Rev. SDP/MultiArch/2016.2/20755 (2016/09/29) Jobs Fixed since change 20559 up to 20755 (F=Feature, B=Bug): job000545 (F): Add option to install_sdp_python.sh to build P4Python with SSL. job000595 (B): Fix attempted login count in p4login. job000597 (B): Fixed order-of-operations issue that caused some log files to be lost on rotation. job000598 (B): Cosmetic fix to broker init script with 'status all' run w/no config files. job000600 (F): Add Helix Management System (HMS) files to SDP. job000601 (F): Enhance p4login to avoid hang issue with auth.id set. job000602 (F): Further test suite docker environment updates. job000603 (F): Add Two Factor Authentication sample auth script. job000604 (B): Correct crontab to run sync_replica.sh every day. job000605 (B): Fix bugs in convert_label_to_autoreload.py. Jobs Fixed since change 20491 up to 20559 (F=Feature, B=Bug): job000590 (B): Fix recent regression with 'restart' option in init scripts. Released: Rev. SDP/MultiArch/2016.1/20491 (2016/09/09) Jobs Fixed since change 20460 up to 20491 (F=Feature, B=Bug): job000589 (B): Fix special case env insulation bug running init scripts as root. Released: Rev. SDP/MultiArch/2016.1/20460 (2016/09/08) Jobs Fixed since change 20395 up to 20460 (F=Feature, B=Bug): job000584 (B): Fixed a typo inserted in mkdirs.sh. job000588 (B): chmod +x fix on some scripts. Released: Rev. SDP/MultiArch/2016.1/20395 (2016/09/04) Jobs Fixed between changelists 20387 and 20395 (F=Feature, B=Bug): job000584 (B): Fixed a typo inserted in mkdirs.sh. Released: Rev. SDP/MultiArch/2016.1/20387 (2016/09/04) Jobs Fixed between changelists 20348 and 20387 (F=Feature, B=Bug): job000579 (B): Fixed several issues in email.sh maint script. job000582 (B): Fixed using with runRemoteCmd getting error if USER not defined in non-core SDP script. job000583 (B): Removed references to legacy names for checkpoint scripts (non-functional change). Jobs Fixed between changelists 20028 and 20348 (F=Feature, B=Bug): job000545 (F): Add option to install_sdp_python.sh to build P4Python with SSL. job000564 (F): Add p4sanity_check.sh script from Battle School. job000565 (B): Fix missing echo of command into log file in p4verify.sh. job000567 (F): Change adminspass to use hidden, and now instance-specific, password files job000568 (F): Add mkrep script to simplify replica creation. job000569 (F): Update p4verify.sh to verify shelves on master/replicas. job000570 (F): Add script to convert labels to autoreload. job000571 (F): Add Evil Twin Detector maintenance script. job000575 (F): Add script to simplify adoption of OS-level best practices recommended by Support. job000577 (B): Eliminate syntax error if user forgets start/stop/status arg; just display status. job000578 (F): Use pid to shutdown p4d rather than 'p4 admin stop' to enhance reliability. Released: Rev. SDP/MultiArch/2016.1/20028 (2016/08/03) Jobs Fixed between changelists 19888 and 20028 (F=Feature, B=Bug): job000548 (F): Defend against environment hijacking of P4ENVIRO. job000557 (F): Revive external auth LDAP group mirror scripts. job000550 (B): sdp_sync.sh v2.1.7: Fixed environment bug (missing defs), HMS support. job000553 (F): Simplify login ticket handling for sophisticated topologies. job000556 (F): Prevent cross-instance contamination if p4_vars is loaded multiple times. Released: Rev. SDP/MultiArch/2016.1/19888 (2016/07/07) Jobs Fixed between changelists 19838 and 19888 (F=Feature, B=Bug): job000274 (B): SDP trigger breaks with Streams workspaces in P4V. job000547 (B): Daily/weekly_backup can fail leaving lock file present requiring manual intervention job000548 (F): Defend against environment hijacking of P4ENVIRO. Released: Rev. SDP/MultiArch/2016.1/19838 (2016/06/26) Jobs Fixed between changelists 19768 and 19838 (F=Feature, B=Bug): job000274 (B): SDP triggers break with Streams workspaces in P4V. job000546 (F): Add defense against known-misbehaving client versions. Released: Rev. SDP/MultiArch/2016.1/19768 (2016/06/24) Jobs Fixed between changelists 19661 and 19768 (F=Feature, B=Bug): job000541 (F): Change to best-practice configurables; auth.autologinprompt=0. job000543 (B): p4master_run operates incorrectly as root. Released: Rev. SDP/MultiArch/2016.1/19661 (2016/06/08) Jobs Fixed between changelists 19410 and 19661 (F=Feature, B=Bug): job000538 (F): Add KEEPJNLS to allow keeping more jnls than ckps. job000539 (F): Rename replica scripts to be consistent with earlier rename of checkpoint scripts. job000540 (F): Enhance error checkong and logging replica handling scripts. job000541 (F): Change to best-practice configurables; auth.autologinprompt=0. Released: SDP/MultiArch/2016.1/19410 (2016/05/17) Jobs Fixed between changelists 18958 and 19410: job000510: Work with license check in P4D 2016.1. job000512: Change p4verify.sh to use -S to verify shelves on a replica instead job000513: Removed the HOST_IP settings from mkdirs and instance_vars since it job000514: Dockerify SDP test suite. job000515: Add replica cleanup script on Windows. job000516: Get p4verify.sh to check SHAREDDATA on replicas. job000517: Avoid chmod on a replica if it shares data with the master. job000518: Use 'p4 admin journal' on master to rotate journal. Released: SDP/MultiArch/2016.1/18958 (2016/04/08) Jobs Fixed between changelists 18619 and 18958: job000484: Proxy status init script can show proxy is up when it is not. job000489: Moved ckp_running.txt to LOGS dir. job000490: Fix missing LOGFILE= in replica_cleanup.sh. job000491: De-emphasized changeType=restricted. job000492: Update Perl/P4Perl Python/P4Python job000493: Support multiple broker configs. job000494: Enhanced p4_vars.template to support non-p4d hosts. job000495: SDP-ify P4DTG init script. job000496: Enhanced p4deleteuser.py to check for shelves, double-deletes. Released: SDP/MultiArch/2016.1/18619 (2016/03/10): Jobs Fixed between changelists 17297 and 18619 job000427 on 2015/12/15 by ttyler *closed* 'Enhanced offline_db 'stale data' Jobs Fixed between changelists 16784 and 17297: job000436: Support $MAILTO values with'#' char in gen_default_broker_cfg.sh. job000446: sdp_sync.sh: Fix 'null crontab' bug. job000447: p4d_base: When stopping p4d, send stderr to /dev/null. job000448: global_replica_status.sh: Support instance-specifc host config. job000449: Support email addresses starting with '#'. Released: SDP/MultiArch/2016.1/16784 (2016/01/12) job000434: Add documentation for sdp_sync.sh to SDP Guide. job000436: Support $MAILTO values with'#' char in gen_default_broker_cfg.sh. -------------------------------------------------------------------------- For more information on jobs, change history details, etc, see: https://swarm.workshop.perforce.com/projects/perforce-software-sdp