sdp_upgrade.sh.man.txt #20

USAGE for sdp_upgrade.sh v2.3.0:

   sdp_upgrade.sh [-y] [-p] [-L <log>|off] [-D]

   or

sdp_upgrade.sh -h|-man

This script must be executed from the 'sdp_upgrade' directory in the extracted
SDP tarball, which can be in one of two locations.  If the /opt/perforce/helix-sdp/sdp
directory exists, start the upgrade process like this (as the 'root' user):

  cd /opt/perforce/helix-sdp/sdp/Server/Unix/p4/common/sdp_upgrade
  ./sdp_upgrade.sh

Otherwise, start the upgrade like this, as the operating system user account
under which the p4d service runs as (e.g. 'perforce' or 'p4admin' but never
as 'root'):

  cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
  ./sdp_upgrade.sh

Running sdp_upgrade.sh without '-y' will do a dry run.  Review the output and check
for a SUCCESS indication at the end.  If you get the success message, proceedd
with:

  ./sdp_upgrade.sh -y


DESCRIPTION:

	This script upgrades Perforce Helix Server Deployment Package (SDP) from
	SDP 2020.1 to the version included in the latest SDP version, SDP
	2024.2.

	== Pre-Upgrade Planning ==

	This script will upgrade the SDP if the pre-upgrade starting SDP version
	is SDP 2020.1 or later, including any/all patches of SDP
	2020.1.

	If the current SDP version is older than 2020.1, it must first be upgraded
	to SDP 2020.1 using the SDP Legacy Upgrade Guide.  For upgrading from
	pre-20.1 versions dating back to 2007, in-place or migration-style upgrades
	can be done.  See:

	https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Legacy_Upgrades.Unix.html

	The SDP should always be upgraded to the latest version first before
	Helix Core binaries p4d/p4broker/p4p are upgraded using the SDP
	upgrade.sh script.

	Upgrading the SDP first ensures the version of the SDP you have
	is compatible with the latest versions of p4d/p4broker/p4p/p4, and
	will always be compatible with all supported versions of these
	Helix Core binaries.

	When this script is used, i.e. when the current SDP version is 2020.1
	or newer, the SDP upgrade procedure does not require downtime for any
	running Perforce Helix services, such as p4d, p4broker, or p4p.  This
	script is safe to run in environments where live p4d instances are running,
	and does not require p4d, p4broker, p4p, or any other services to be stopped
	or upgraded.  Upgrade of the SDP is cleanly separate from the upgrade the
	Helix Core binaries. The upgrade of the SDP can be done immediately prior to
	Helix Core upgrades, or many days prior.

	There can be multiple SDP instances on a given server machine. This script
	will upgrade the SDP on the machine, and thus after the upgrade all
	instances will immediately use new SDP scripts and updated instance
	configuration files, e.g. the /p4/common/config/p4_N.vars files.  However,
	all instances will continue running the same Helix Core binaries.  Any live
	running Helix Core server process on the machine are unaffected by the
	upgrade of SDP.

	This script will upgrade the SDP on a single machine. If your Perforce
	Helix topology has multiple machines, the SDP should be upgraded on all
	machines. The upgrade of SDP on multiple machines can be done in any order,
	as there is no cross-machine dependency requiring the SDP to be the same
	version.  (The order of upgrade of Helix Core services and binaries such as
	p4d in global topologies with replicas and edge servers does matter, but is
	outside the scope of this script).

	Planning Recap:
	1. The SDP can be upgraded without downtime when this script is used, i.e.
	when the starting SDP version is 2020.1 or later.
	2. Upgrade SDP on all machines, in any order, before upgrading p4d and other
	Helix binaries.

	== Diretory Structure Changes for /p4/sdp and /p4/common ==

	There is a structure change with SDP affecting where the /p4/sdp and
	/p4/common symlinks are targeted. This change is part of a phased
	rollout of a new structure to be used by a future helix-sdp OS package.
	Nothing in the structural changes affects behaviors of routine
	SDP daily scripts. The changes affect how SDP upgrades work, and on
	what volumes files like Helix Core server binaries exist on.  The gist
	of the change is that that /hxdepots/p4/commnon and /hxdepots/sdp
	folders (which are on NFS if /hxdepots is NFS-mounted) are changeed from
	being actively used folders to become backup directories,

	The active directories are moved to local storage on the machine
	in the new /opt/perforce/helix-sdp structure.

	Prior to SDP 2024.2, this was the typical symlink structure is:
       
	* /hxdepots/sdp, symlinked as /p4/sdp
	* /hxdepots/p4/common, symlinked as /p4/common
	* /hxdepots/downloads, no symlnk

	If /hxdepots is NFS mounted, then the active folders are on NFS.

	Starting with 2024.2, a new structure is available, and will be
	used if the install_sdp.sh script was used for the initial SDP
	install.  In that structure, we have the following:

	* /opt/perforce/helix-sdp/p4/sdp, symlinked as /p4/sdp
	* /opt/perforce/helix-sdp/p4/common, symlinked as /p4/common
	* /opt/perforce/helix-sdp/downloads, no symlink
	* /opt/perforce/helix-sdp/sdp, immutable root-owned structure,
	updated only by SDP upgrades (i.e. this script).

	During upgrades, the legacy structure is changed in ways that
	are safe even if /hxdepots is NFS-shared with machines not
	being upgraded. For safety, some structures are abandoned and
	untouched.

	* A new /hxdepots/sdp/backup folder is created, and contain
	/hxdepots/sdp/backup/opt/perforce/helix-sdp
	* Dirs other than backup under /helix/sdp are unusued/untouched
	* /hxdepots/p4/common is unused/untouched
	* /hxdepots/downloads is moved to /hxdepots/sdp/backup/downloads,
	and the /hxdepots/sdp becomes a symlink to /opt/perforce/helix-sdp/downloads.

	== NFS Sharing of HxDepots ==

	In some environments, the HxDepots volume is shared across multiple server
	machines with NFS, typically mounted as /hxdepots.  This script updates the
	/hxdepots/p4/common and /hxdepots/sdp directories, both of which are on the
	NFS mount.  Thus upgrading SDP on a single machine will effectively and
	immediately upgrade the SDP on all machines that share /hxdepots from the
	same NFS-mounted storage. This is a safe and valid configuration, as
	upgrading the SDP does not affect any live running p4d servers.

	== Acquiring the SDP Package - OS Package Structure ==

	If the /opt/perforce/helix-sdp structure exists on your machine, then
	upgrade using the procedure in this section.  Otherwise see the section
	below "Acquiring the SDP Package - Classic Structure".

	Become the root user first:
	  sudo su -
	  cd /opt/perforce/helix-sdp/downloads
	  [[ -e sdp.Unix.tgz ]] && mv -f sdp.Unix.tgz sdp.Unix.$(date +'%Y-%m-%d-%H%M%S').tgz
	  curl -L -O https://swarm.workshop.perforce.com/download/guest/perforce_software/sdp/downloads/sdp.Unix.tgz
	  tar -tzf sdp.Unix.tgz 2>&1 | grep -q sdp/Version && echo OK

	If this does not display an OK message, the tarball is not valid. Investigate
	and resolve this issue before proceeding. As root:

	  cd /opt/perforce/helix-sdp
	  [[ -d backup ]] || mkdir backup
	  mv sdp backup/sdp.old.$(date +'%Y-%m-%d-%H%M%S')
	  tar -xzf downloads/sdp.Unix.tgz

	  cd /opt/perforce/helix-sdp/sdp/Server/Unix/p4/common/sdp_upgrade
	  ./sdp_upgrade.sh -man

	== Acquiring the SDP Package - Classic Structure ==

	If the /opt/perforce/helix-sdp structure exists on your machine, then
	upgrade using the procedure above in the OS Package Structure section.

	This script is part of the SDP package (tarball). It must be run from an
	extracted tarball directory.  Acquiring the SDP tarball is a manual operation.

	The SDP tarball must be extracted such that the 'sdp' directory appears as
	<HxDepots>/downloads/new/sdp, where <HxDepots> defaults to /hxdepots. To
	determine the value for <HxDepots> at your site you can run the following:
   
     bash -c 'cd /p4/common; d=$(pwd -P); echo ${d%/p4/common}'
   
   On this machine, that value is: /hxdepots

	Following are sample commands to acquire the latest SDP, to be executed
	as the user perforce:

	  cd /hxdepots
	  [[ -d downloads ]] || mkdir downloads
	  cd downloads
	  [[ -d new ]] && mv new old.$(date +'%Y%m%d-%H%M')
	  curl -L -O https://swarm.workshop.perforce.com/download/guest/perforce_software/sdp/downloads/sdp.Unix.tgz
	  ls -l sdp.Unix.tgz
	  mkdir new
	  cd new
	  tar -xzf ../sdp.Unix.tgz

	After extracting the SDP tarball, cd to the directory where this
	sdp_upgrade.sh script resides, and execute it from there.

	  cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
	  ./sdp_upgrade.sh -man

	== Preflight Checks ==

	Prior to upgrading, preflight checks are performed to ensure the
	upgrade can be completed successfully. If the preflight checks
	fail, the upgrade will not start.

	Sample Preflight Checks:

	* The existing SDP version is verified to be SDP 2020.1+.
	* Various basic SDP structural checks are done.
	* The /p4/common/bin/p4_vars is checked to confirm it can be upgraded.
	* All /p4/common/config/p4_N.vars files are checked to confirm they can be upgraded.

	== Automated Upgrade Processing ==

	Step 1: Backup /p4/common.

	The existing <HxDepots>/p4/common structure is backed up to:

	<HxDepots>/p4/common.bak.<YYYYMMDD-hhmm>

	Step 2: Update /p4/common.

	The existing SDP /p4/common structure is updated with new
	versions of SDP files.

	Step 3: Generate the SDP Environment File.

	Regenerate the SDP general environment file,
	/p4/common/bin/p4_vars.

	The template is /p4/common/config/p4_vars.template.

	Step 4: Generate the SDP Instance Files.

	Regenerate the SDP instance environment files for all instances based on
	the new template.

	The template is /p4/common/config/instance_vars.template.

	For Steps 3 and 4, the re-generation logic will preserve current
	settings. If upgrading from SDP r20.1, any custom logic that
	exists below the '### MAKE LOCAL CHANGES HERE' tag will be
	split into separate files.  Custom logic in p4_vars will be moved
	to /p4/common/site/config/p4_vars.local. Custom logic in
	p4_N.vars files will be moved to /p4/common/site/config/p4_N.vars.local.

	Note: Despite these changes, the mechanism for loading the SDP shell
	environment remains unchanged since 2007, so it looks like:

	$ source /p4/common/bin/p4_vars N

	Changes to the right-side of assignments for specific are preserved
	for all defined SDP settings.  For p4_vars, preserved settings are:
	  - OSUSER (determined by current owner of /p4/common)
	  - KEEPLOGS
	  - KEEPCKPS
	  - KEEPJNLS
	
	For instance_vars files, preserved settings are:
	  - MAILTO
	  - MAILFROM
	  - P4USER
	  - P4MASTER_ID
	  - SSL_PREFIX
	  - P4PORTNUM
	  - P4BROKERPORTNUM
	  - P4MASTERHOST
	  - PROXY_TARGET
	  - PROXY_PORT
	  - PROXY_MON_LEVEL
	  - PROXY_V_FLAGS
	  - P4DTG_CFG
	  - SNAPSHOT_SCRIPT
	  - SDP_ALWAYS_LOGIN
	  - SDP_AUTOMATION_USERS
	  - SDP_MAX_START_DELAY_P4D
	  - SDP_MAX_START_DELAY_P4BROKER
	  - SDP_MAX_START_DELAY_P4P
	  - SDP_MAX_STOP_DELAY_P4D
	  - SDP_MAX_STOP_DELAY_P4BROKER
	  - SDP_MAX_STOP_DELAY_P4P
	  - VERIFY_SDP_SKIP_TEST_LIST
	  - The 'umask' setting.
	  - KEEPLOGS (if set)
	  - KEEPCKPS (if set)
	  - KEEPJNLS (if set)
	
	Note that the above list excludes any values that are calculated.
	
	Step 5: Remove Deprecated Files.

	Deprecated files will be purged from the SDP structure.  The list of
	files to be cleaned are listed in this file:

	/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade/deprecated_files.txt

	Paths listed in this file are relative to the '/p4' directory (or
	more accurately the SDP Install Root directory, which is always
	'/p4' except in SDP test production environments).

	Step 6: Update SDP crontabs.

	No crontab updates are required for this SDP upgrade.
	
	== Post-Upgrade Processing ==

	This script provides guidance on any post-processing steps. For some
	releases, this may include upgrades to crontabs.

OPTIONS:
-y	Specify the '-y' option to confirm that the SDP upgrade should be done.

	By default, this script operates in No-Op mode, meaning no actions
	that affect data or structures are taken.  Instead, commands that would
	be run are displayed.  This mode can be educational, showing various
	steps that will occur during an actual upgrade.

 -p	Specify '-p' to halt processing after preflight checks are complete,
	and before actual processing starts. By default, processing starts
	immediately upon successful completion of preflight checks.

 -Od	Specify '-Od' to override the rule preventing downgrades.

	WARNING: This is an advanced option intended for use by or with the
	guidance of Perforce Support or Perforce Consulting.

 -L <log>
	Specify the log file to use.  The default is /tmp/sdp_upgrade.<timestamp>.log

	The special value 'off' disables logging to a file. This cannot be
	specified if '-y' is used.

 -d	Enable debugging verbosity.

 -D	Set extreme debugging verbosity.

HELP OPTIONS:
 -h	Display short help message
 -man	Display man-style help message

FILES AND DIRECTORIES:
	Name: SDPCommon
	Path: /p4/common
	Notes: This sdp_upgrade.sh script updates files in and under this folder.

	Name: HxDepots
	Default Path: /hxdepots
	Notes: The folder containing versioned files, checkpoints, and numbered
	journals, and the SDP itself. This is commonly a mount point.

	Name: DownloadsDir
	Default Path: /hxdepots/downloads

	Name: SDPInstallRoot
	Path: /p4

	EDITME - Add new structure dirs /opt/perforce/helix-sdp

EXAMPLES:
	This script must be executed from 'sdp_upgrade' directory in the extracted
	SDP tarball.  Typical operation starts like this:

	cd /hxdepots/downloads/new/sdp/Server/Unix/p4/common/sdp_upgrade
	./sdp_upgrade.sh -h

	All following examples assume operation from that directory.

	Example 1: Prelight check only:

	sdp_upgrade.sh -p

	Example 2: Preview mode:

	sdp_upgrade.sh

	Example 3: Live operation:

	sdp_upgrade.sh -y

LOGGING:
	This script generates a log file, ~/sdp_upgrade.<timestamp>.log
	by default. See the '-L' option above.

CUSTOM PRE- AND POST- UPGRADE AUTOMATION HOOKS:
	This script can execute custom pre- and post- upgrade scripts. This
	can be useful to incorporate site-specific elements of an SDP upgrade.

	If the file /p4/common/site/upgrade/pre-sdp_upgrade.sh exists and is
	executable, it will be executed as a pre-upgrade script. If the file
	/p4/common/site/upgrade/post-sdp_upgrade.sh exists and is executable,
	it will be executed as a post-upgrade script.
	
	Pre- and post- upgrade scripts are passed the '-y' flag to confirm
	actual processing is to be done.  Custom scripts are expected to
	operate in preview mode by default, taking no actions that affect data
	(just as this script behaves).  If this sdp_upgrade.sh script is given
	the '-y' flag, that option is passed to the custom script as well,
	indicating active processing should occur.

	Pre- and post- upgrade scripts are expected to exit with a zero exit
	code to indicate success, and non-zero to indicate failure.
	The custom pre-upgrade script is executed after standard preflight
	checks complete successfully.  Preflight checks are executed before
	actual upgrade processing commences. If a custom pre-upgrade script
	indicates a failure, the overall upgrade process aborts.

	The post-upgrade custom script is executed after the main SDP upgrade
	is successful.

	Success or failure of pre- and post- upgrade scripts is reported in
	the log.  These scripts do not require independent logging, as all
	standard and error output is captured in the log of this sdp_upgrade.sh
	script.

	TIP: Be sure to fully test custom scripts in a test environment
	before incorporating them into an upgrade on production systems.

EXIT CODES:
	An exit code of 0 indicates no errors were encountered during the
	upgrade. A non-zero exit code indicates the upgrade was aborted
	or failed.