USAGE for sdp_upgrade.sh v1.7.6: sdp_upgrade.sh [-y] [-p] [-L |off] [-D] or sdp_upgrade.sh -h|-man This script must be executed from the 'sdp_upgrade' directory in the extracted SDP tarball. Typical operation starts like this: cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade ./sdp_upgrade.sh -h DESCRIPTION: This script upgrades Perforce Helix Server Deployment Package (SDP) from SDP 2020.1 to the version included in the latest SDP version, SDP 2022.2. == Pre-Upgrade Planning == This script will upgrade the SDP if the pre-upgrade starting SDP version is SDP 2020.1 or later, including any/all patches of SDP 2020.1. If the current SDP version is older than 2020.1, it must first be upgraded to SDP 2020.1 using the SDP Legacy Upgrade Guide. For upgrading from pre-20.1 versions dating back to 2007, in-place or migration-style upgrades can be done. See: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Legacy_Upgrades.Unix.html The SDP should always be upgraded to the latest version first before Helix Core binaries p4d/p4broker/p4p are upgraded using the SDP upgrade.sh script. Upgrading the SDP first ensures the version of the SDP you have is compatible with the latest versions of p4d/p4broker/p4p/p4, and will always be compatible with all supported versions of these Helix Core binaries. When this script is used, i.e. when the current SDP version is 2020.1 or newer, the SDP upgrade procedure does not require downtime for any running Perforce Helix services, such as p4d, p4broker, or p4p. This script is safe to run in environments where live p4d instances are running, and does not require p4d, p4broker, p4p, or any other services to be stopped or upgraded. Upgrade of the SDP is cleanly separate from the upgrade the Helix Core binaries. The upgrade of the SDP can be done immediately prior to Helix Core upgrades, or many days prior. There can be multiple SDP instances on a given server machine. This script will upgrade the SDP on the machine, and thus after the upgrade all instances will immediately use new SDP scripts and updated instance configuration files, e.g. the /p4/common/config/p4_N.vars files. However, all instances will continue running the same Helix Core binaries. Any live running Helix Core server process on the machine are unaffected by the upgrade of SDP. This script will upgrade the SDP on a single machine. If your Perforce Helix topology has multiple machines, the SDP should be upgraded on all machines. The upgrade of SDP on multiple machines can be done in any order, as there is no cross-machine dependency requiring the SDP to be the same version. (The order of upgrade of Helix Core services and binaries such as p4d in global topologies with replicas and edge servers does matter, but is outside the scope of this script). Planning Recap: 1. The SDP can be upgraded without downtime when this script is used, i.e. when the starting SDP version is 2020.1 or later. 2. Upgrade SDP on all machines, in any order, before upgrading p4d and other Helix binaries. == NFS Sharing of HxDepots == In some environments, the HxDepots volume is shared across multiple server machines with NFS, typically mounted as /hxdepots. This script updates the /hxdepots/p4/common and /hxdepots/sdp directories, both of which are on the NFS mount. Thus upgrading SDP on a single machine will effectively and immediately upgrade the SDP on all machines that share /hxdepots from the same NFS-mounted storage. This is a safe and valid configuration, as upgrading the SDP does not affect any live running p4d servers. == Acquiring the SDP Package == This script is part of the SDP package (tarball). It must be run from an extracted tarball directory. Acquiring the SDP tarball is a manual operation. The SDP tarball must be extracted such that the 'sdp' directory appears as /downloads/new/sdp, where defaults to /hxdepots. To determine the value for at your site you can run the following: bash -c 'cd /p4/common; d=$(pwd -P); echo ${d%/p4/common}' On this machine, that value is: /hxdepots Following are sample commands to acquire the latest SDP, to be executed as the user perforce: cd /hxdepots [[ -d downloads ]] || mkdir downloads cd downloads [[ -d new ]] && mv new old.$(date +'%Y%m%d-%H%M') curl -s -k -O https://swarm.workshop.perforce.com/projects/perforce-software-sdp/download/downloads/sdp.Unix.tgz mkdir new cd new tar -xzf ../sdp.Unix.tgz After extracting the SDP tarball, cd to the directory where this sdp_upgrade.sh script resides, and execute it from there. cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade ./sdp_upgrade.sh -man == Preflight Checks == Prior to upgrading, preflight checks are performed to ensure the upgrade can be completed successfully. If the preflight checks fail, the upgrade will not start. Sample Preflight Checks: * The existing SDP version is verified to be SDP 2020.1+. * Various basic SDP structural checks are done. * The /p4/common/bin/p4_vars is checked to confirm it can be upgraded. * All /p4/common/config/p4_N.vars files are checked to confirm they can be upgraded. == Automated Upgrade Processing == Step 1: Backup /p4/common. The existing /p4/common structure is backed up to: /p4/common.bak. Step 2: Update /p4/common. The existing SDP /p4/common structure is updated with new versions of SDP files. Step 3: Generate the SDP Environment File. Regenerate the SDP general environment file, /p4/common/bin/p4_vars. The template is /p4/common/config/p4_vars.template. Step 4: Generate the SDP Instance Files. Regenerate the SDP instance environment files for all instances based on the new template. The template is /p4/common/config/instance_vars.template. For Steps 3 and 4, the re-generation logic will preserve current settings. If upgrading from SDP r20.1, any custom logic that exists below the '### MAKE LOCAL CHANGES HERE' tag will be split into separate files. Custom logic in p4_vars will be moved to /p4/common/site/config/p4_vars.local. Custom logic in p4_N.vars files will be moved to /p4/common/site/config/p4_N.vars.local. Note: Despite these changes, the mechanism for loading the SDP shell environment remains unchanged since 2007, so it looks like: $ source /p4/common/bin/p4_vars N Changes to the right-side of assignments for specific are preserved for all defined SDP settings. For p4_vars, preserved settings are: - OSUSER (determined by current owner of /p4/common) - KEEPLOGS - KEEPCKPS - KEEPJNLS For instance_vars files, preserved settings are: - MAILTO - MAILFROM - P4USER - P4MASTER_ID - SSL_PREFIX - P4PORTNUM - P4BROKERPORTNUM - P4MASTERHOST - PROXY_TARGET - PROXY_PORT - PROXY_V_FLAGS - P4DTG_CFG - SNAPSHOT_SCRIPT - SDP_ALWAYS_LOGIN - SDP_AUTOMATION_USERS - SDP_MAX_START_DELAY_P4D - SDP_MAX_START_DELAY_P4BROKER - SDP_MAX_START_DELAY_P4P - SDP_MAX_STOP_DELAY_P4D - SDP_MAX_STOP_DELAY_P4BROKER - SDP_MAX_STOP_DELAY_P4P - VERIFY_SDP_SKIP_TEST_LIST - The 'umask' setting. - KEEPLOGS (if set) - KEEPCKPS (if set) - KEEPJNLS (if set) Note that the above list excludes any values that are calculated. Step 5: Remove Deprecated Files. Deprecated files will be purged from the SDP structure. The list of files to be cleaned are listed in this file: /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade/deprecated_files.txt Paths listed in this file are relative to the '/p4' directory (or more accurately the SDP Install Root directory, which is always '/p4' except in SDP test production environments). Step 6: Update SDP crontabs. No crontab updates are required for this SDP upgrade. == Post-Upgrade Processing == This script provides guidance on any post-processing steps. For some releases, this may include upgrades to crontabs. OPTIONS: -y Specify the '-y' option to confirm that the SDP upgrade should be done. By default, this script operates in No-Op mode, meaning no actions that affect data or structures are taken. Instead, commands that would be run are displayed. This mode can be educational, showing various steps that will occur during an actual upgrade. -p Specify '-p' to halt processing after preflight checks are complete, and before actual processing starts. By default, processing starts immediately upon successful completion of preflight checks. -L Specify the log file to use. The default is /tmp/sdp_upgrade..log The special value 'off' disables logging to a file. This cannot be specified if '-y' is used. -d Enable debugging verbosity. -D Set extreme debugging verbosity. HELP OPTIONS: -h Display short help message -man Display man-style help message FILES AND DIRECTORIES: Name: SDPCommon Path: /p4/common Notes: This sdp_upgrade.sh script updates files in and under this folder. Name: HxDepots Default Path: /hxdepots Notes: The folder containing versioned files, checkpoints, and numbered journals, and the SDP itself. This is commonly a mount point. Name: DownloadsDir Default Path: /hxdepots/downloads Name: SDPInstallRoot Path: /p4 EXAMPLES: This script must be executed from 'sdp_upgrade' directory in the extracted SDP tarball. Typical operation starts like this: cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade ./sdp_upgrade.sh -h All following examples assume operation from that directory. Example 1: Prelight check only: sdp_upgrade.sh -p Example 2: Preview mode: sdp_upgrade.sh Example 3: Live operation: sdp_upgrade.sh -y LOGGING: This script generates a log file, ~/sdp_upgrade..log by default. See the '-L' option above. CUSTOM PRE- AND POST- UPGRADE AUTOMATION HOOKS: This script can execute custom pre- and post- upgrade scripts. This can be useful to incorporate site-specific elements of an SDP upgrade. If the file /p4/common/site/upgrade/pre-sdp_upgrade.sh exists and is executable, it will be executed as a pre-upgrade script. If the file /p4/common/site/upgrade/post-sdp_upgrade.sh exists and is executable, it will be executed as a post-upgrade script. Pre- and post- upgrade scripts are passed the '-y' flag to confirm actual processing is to be done. Custom scripts are expected to operate in preview mode by default, taking no actions that affect data (just as this script behaves). If this sdp_upgrade.sh script is given the '-y' flag, that option is passed to the custom script as well, indicating active processing should occur. Pre- and post- upgrade scripts are expected to exit with a zero exit code to indicate success, and non-zero to indicate failure. The custom pre-upgrade script is executed after standard preflight checks complete successfully. Preflight checks are executed before actual upgrade processing commences. If a custom pre-upgrade script indicates a failure, the overall upgrade process aborts. The post-upgrade custom script is executed after the main SDP upgrade is successful. Success or failure of pre- and post- upgrade scripts is reported in the log. These scripts do not require independent logging, as all standard and error output is captured in the log of this sdp_upgrade.sh script. TIP: Be sure to fully test custom scripts in a test environment before incorporating them into an upgrade on production systems. EXIT CODES: An exit code of 0 indicates no errors were encountered during the upgrade. A non-zero exit code indicates the upgrade was aborted or failed.