reset_sdp.command_summary.txt #28

USAGE for reset_sdp.sh v4.6.3:

reset_sdp.sh [-c <cfg>] [-no_ssl] [-no_sd] [-no_cron] [-no_ppr] [-no_systemd] [-no_tweaks] [-ls] [-v] [-fast|-dapi] [-t <ServerType>] [-s <ServerID>] [-ts <TargetServerID>] [-tp <TargetPort>] [-lp <ListenPort>] [-se] [-H <hostname>] [-T <timezone>] [-local|-B] [[-d <sdp_dir>] | [-b <branch>[,@cl]]] [-p <dir1>[,<dir2>,...]>] [-i <helix_installer_branch>] [-D] [-X|-R] [-M]

or

reset_sdp.sh -C > settings.cfg

or

reset_sdp.sh [-h|-man]


SAFETY NOTICE:
	This script SHOULD NEVER EXIST on a Production Perforce server.

DESCRIPTION:
	This script simplifies the process of testing an SDP installation,
	repetitively blasting all process by the 'perforce' user and resetting
	the SDP from the ground up, blasting typical SDP folders each time.

	It installs the Perforce Helix Core server (P4D) with a P4Broker,
	and installs the Perforce Sample Depot data set used for training
	and PoC installations.

	It is helpful when bootstrapping a demo server with a sample data
	set, complete with broker, and optionally Perl/P4Perl and
	Python/P4Python.

	This script handles many aspects of installation. It does the
	following:
	* Creates the OS user that will run the Helix Core p4d process,
	  the 'perforce' user by default, using the 'useradd' command,
	  unless that account already exists.  If a non-local account
	  is to be used, that should be created first before running this
	  script. If the account is created using 'useradd', the password
	  will be set to match that of the admin P4USER, which is also
	  'perforce' by default (matching the OSUSER).
	* Creates the home directory for the OSUSER user, if needed.
	* Adds OS packages as needed for P4Perl/P4Python local builds
	  (if -dapi is specified).

	Following installation, it also does the following to be more
	convenient for demos, and also give a more production-like feel:
	* Grants the perforce user sudo access (full or limited).
	* Creates default ~perforce/.bash_profile and .bashrc files.
	* Connects to the Perforce Package Repository (APT and YUM only).
	* Adds firewalld rules for Helix server and broker ports ('firewalld'
	only; there is no support for the 'ufw' or other firewalls).
	* Installs crontab for ~perforce user.

PLATFORM SUPPORT:
	This works on Red Hat Enterprise Linux, CentOS, and Mac OSX
	10.10+ thru Mojave platforms.  It works on RHEL/CentOS
	6.4-7.6, SuSE Linux 12, and likely on Ubuntu 18 and other Linux
	distros with little or no modification.

	This script currently supports the bin.linux26x86_64 (Linux) and
	bin.maxosx1010x86_64 (Mac OSX/Darwin) architectures.

	This script recognizes SysV, Systemd, and Launchd init mechanisms,
	though does not currently support Launchd on OSX.

	For Mac OSX, note that this requires bash 4.x, and the default
	bash on Mac OSX remains 3.x as of OSX Mojave.  For operating on
	Mac, the /bin/bash shebang line needs to be adjusted to reference
	a bash 4 version, e.g. /usr/local/bin/bash if installed with
	Homebrew.

REQUIREMENTS:
	The following OS packages are installed (unless '-fast' is
	used):

	* Yum: cronie curl gcc gcc-c++ mailx make openssl openssl-devel rsync tar wget which zlib zlib-devel

	* AptGet: build-essential cron libssl-dev make zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev rsync

	* Zypper: cronie curl gcc gcc-c++ make openssl openssl-devel wget which zlib zlib-devel rsync

	Development utilities such as 'make', the 'gcc' compiler,
	and 'curl' must be installed and available in the PATH (unless
	running with '-fast').  The package installation as noted
	above ensures these dependencies are available.

OPTIONS:
 -c <cfg>
	Specify a config file.  By default, values for various settings
	such as the email to send script logs to are configure with
	demo values, e.g. P4AdminList@p4demo.com.  Optionally, you can
	specify a config file to define your own values.

	For details on what settings you can define in this way, run:
	reset_sdp.sh -C > setings.cfg

	Then modify the generated config file settings.cfg as desired.
	The generated config file contains documentation on settings and
	values.  If no changes are made to the generated file, running with
	'-c settings.cfg' is the equivalent of running without using '-c' at
	all.

 -C	See '-c <cfg>' above.

 -no_ssl
	By default, the Perforce server is setup SSL-enabled.  Specify
	'-no_ssl' to avoid using SSL feature.

 -no_sd
	By default, the Perforce Sample Depot data set is loaded.
	Specify '-no_sd' to skip loading the Sample Depot.

 -no_cron
	Skip initialization of the crontab.

 -no_ppr
	Skip addition of the Perforce Package Repository for YUM/APT
	repos.  By default, the Package Repository is added.

 -ls	Specify that only limited sudo is to be granted.  By default
 	full sudo access is granted to the OSUSER by adding this file:

	/etc/sudoers.d/perforce

	with these contents:

	perforce ALL=(ALL) NOPASSWD: ALL

	If '-ls' is specified, limited sudoers access is provisioned,
	with just enough access to execute commands like:

	systemctl <start|stop|status> p4d_*
	systemctl <start|stop|status> p4dtg_*
	systemctl <start|stop|status> p4broker_*
	systemctl <start|stop|status> p4p_*

	For more detail, see the template file: perforce_sudoers.t

 -v
 	Specify '-v' to run the verify_sdp.sh script after the SDP
	installation is complete. If '-v' is specified and the
	verify_sdp.sh script is available in the SDP, it is executed.
	If the Sample Depot is loaded, the '-online' flag to the
	verify_sdp.sh script is added.  If '-no_cron' is specified,
	the corresponding '-skip cron' option is added verify_sdp.sh.
	If '-no_sd' is specified, the '-skip' tests also exclude
	the offline_db and p4t_files checks in verify_sdp.sh.

 -fast	Specify '-fast' to skip package installation using
	the package manager (yum, apt-get, or zypper).

	The '-fast' flag should not be used if you plan to
	deploy or develop triggers that use P4Python or P4Perl,
	such as the SDP CheckCaseTrigger.py.

 -dapi	Specify '-dapi' (derived API) to attempt install of
	the SDP derived APIs if they are available in the
	SDP package.

	Without the '-dapi' option, installation takes only a
       	few minutes typically.  With '-dapi', expect the
	install to take 20+ minutes due to the time to do
	builds and run automated tests suites for Perl, Python,
	P4Perl, and P4Python.

	The '-fast' and '-dapi' options are mutually exclusive.

	If '-dapi' is not used and a need arises afterward to
	install the derived APIs, that can be done with by running
	the following as the configured OSUSER ('perforce' by default):

	cd /p4/sdp/Server/Unix/setup
	./install_sdp_python.sh
	./install_sdp_perl.sh

	Note: These install_sdp_*.sh scripts may not be available in
	the SDP Package. Also, required packages such as gcc/g++ may
	need to be installed separately

 -local
	By default, various files and binaries are downloaded from
	the Perforce Workshop and the Perforce FTP server as needed.
	If the server machine on which the Helix Installer is to be
	run cannot reach the public internet or if using files from
	external sites is not desired, the '-local' flag can be used.

	With '-local', needed files must be acquired and put in place
	on the server machine on which this script is to be run.  Any
	missing files result in error messages.

	The '-local' argument cannot be used with -B.

	For '-local' to work, the following must exist:

	1. Helix Binaries
	
	Helix binaries must exist in /hxdepots/helix_binaries:

	* /hxdepots/helix_binaries/p4
	* /hxdepots/helix_binaries/p4d
	* /hxdepots/helix_binaries/p4broker
	* /hxdepots/helix_binaries/p4p

	2. Server Deployment Package (SDP)

	The SDP tarball must be acquired an put in place here:

	* /hxdepots/downloads/sdp.Unix.tgz

	3. Sample Depot Tarball
	
	The Sample Depot appropriate to your platform must exist unless the
	the '-no_sd' ('No Sample Depot') is used.

	* /hxdepots/downloads/sampledepot.mac.tar.gz (on Mac/OSX if case-insensitive)
	* /hxdepots/downloads/sampledepot.tar.gz (on UNIX/Linux or case-sensitive Mac)

	4. Helix Installer

	With '-local', the 'reset_sdp.sh' script and all related files must
	be acquired and placed in /hxdepots/reset.

	See EXAMPLES below for sample of acquiring files for use with
	'-local' mode.

 -B	Specify '-B' to blast base SDP dirs, for a clean start.

	Otherwise without '-B', downloaded components from earlier
	runs will be used (which should be fine if they were run
	recently).

	The '-B' flag also replaces files in the /hxdepots/reset
	directory, where this script lives, with those downloaded
	from The Workshop (the versions of which are affected
	by the '-i <helix_installer_branch>' flag, described
	below).

	The '-B' flag also blasts the /tmp/downloads and /tmp/p4perl
	directories, used by install_sdp_python.sh and
	install_sdp_perl.sh, if they exist.

 -no_systemd
	Specify '-no_systemd' to avoid using systemd, even if it
	appears to be available. By default, systemd is used if it
	appears to be available.

	This is helpful in operating in containerized test environments
	where systemd is not available.

 -no_tweaks
 	Skip execution of the SDP operating system tweaks script,
	os_tweaks.sh.

 -t <ServerType>
 
 	Specify the type of server. Valid values are:

	* p4d_master - A master/commit server.
	* p4d_replica - A replica with all metadata from the master (not
	  filtered i any way).
	* p4d_filtered_replica - A filtered replica or filtered forwarding
	  replica.
	* p4d_edge - An edge server.
	* p4d_edge_replica - Replica of an edge server. Also set
	  TargetServerID.
	* p4broker - An SDP host running only a p4broker, with no p4d.
	* p4proxy - An SDP host running only a p4proxy, with no p4d no p4d.
	
 -s <ServerID>
 	Specify the ServerID.  A ServerID is required if the ServerType is
       	any p4d_* type other than p4d_master.

 -ts <TargetServerID>
 	Specify the Target ServerID. Set this only if ServerType is
	p4d_edge_replica. The value is the ServerID of edge server that
	this server is a replica of, and must match the ReplicatingFrom:
	field of the server spec.

 -tp <TargetPort>
 	Specify the target port.  For p4broker and p4proxy only.

 -lp <ListenPort>
 	Specify the port to listen on.  For p4broker and p4proxy only.

 -se
 	Specify -se to simulate email. This generates a mail simlator
	script: /p4/common/site/bin/mail

 -H <hostname>
 	Set the hostname.  This is only supported on systems that
	support the 'hostnamectl' command. The hostname is set by
	doing: hostnamectl set-hostname <hostname>

	If the corresponding 'Hostname' setting is defined in the
	configuration file and this '-H <hostname>' flag is used,
	the command line option will override the config file.

 -T <timezone>
 	Set the timezone.  This is only supported on systems that
	support the 'timedatectl' command. The timezone is set by
	doing: timedatectl set-timezone <timezone>

	If the corresponding 'Timezone' setting is defined in the
	configuration file and this '-T <timezone>' flag is used,
	the command line option will override the config file.

 -p <dir1>[,<dir2>,...]>]
	Specify a comma-delimited list of directories under /p4/common
	to preserve that would otherwise be removed.  Directories must
	be specified as paths relative to /p4/common, and cannot contain
	spaces.

	For example, the value '-p config,bin/triggers' would preserve the
	/p4/common/config and /p4/common/bin/triggers directories.

	Directories specified are moved aside to a temporary working area
	before the SDP folders are removed.  After installation, they are
	moved back via an 'rsync' command with no '--delete' option.  This
	means any files that overlap with the stock install are replaced
	by ones that originally existed, but non-overlapping files are not
	removed.

	This is intended to be useful for developing test suites that
	install server under /p4/common/bin, e.g. Component Based Development
	scripts which install under /p4/common/bin/cbd would use '-p bin/cbd'.

 -d <sdp_dir>
	Specify a directory on the local host containing the SDP to deploy.

	Use the special value '-d default' to use the /sdp directory (as per
	the Docker-based SDP Test Suite environment).

	The directory specified by '-d' is expected to contain either:
	* an SDP tarball (sdp.Unix.tgz) file, or
	* an already-extracted SDP directory, which must include the SDP
	Version file.

 -b <branch>[,@cl]
	The default SDP install method is to use the latest released SDP
	tarball representing the main branch in The Workshop (public.perforce.com:1666).

	The latest tarball can be found on this server, consistently named
	sdp.Unix.tgz. This file appears alongside a version-tagged file
	named something like sdp.Unix.2019.2.25938.tgz.  These appear here:
	https://swarm.workshop.perforce.com/projects/perforce-software-sdp/files/downloads

	Specify '-b' to use a different branch, typically '-b dev'.  This
	changes the install method from a tarball to using a 'p4 clone'
	command using Helix native DVCS features to fetch the very latest
	unreleased files from the branch at any point in time. This is mainly
	of interest when testing the SDP or previewing specific up and
	coming features.

	If '-b' is specified with the optional @cl syntax, where @cl
	specifies a changelist with files shelved on the given branch,
	a form of unshelving is done, enabling a workflow for testing
	shelved changes with the Helix Installer.  So for example,
	specify '-b dev,@23123' to clone from the dev branch, and then
	followed by a fetch of shelved changelist @23123, which is
	expected to have files shelved in the dev branch.

DEBUGGING OPTIONS:
 -i	<helix_installer_branch>

	Specify the branch of the Helix Installer to use.  This affects the
	URL from which Helix Installer files in /hxdepots/reset are pulled from
	The Workshop.  The default is main; an alternative is '-i dev'.

 -D     Set extreme debugging verbosity.

 -X	Extreme reset. This removes the user account for the configured
	OSUSER ('perforce' by default) and blasts all SDP-related directories
	at the start of script operations, including the home directory
	for the configured OSUSER.

	This also clears firewall rules ('firewalld' only).

	Using '-X' does not blast the Helix Installer downloads or
	helix_binaries directories, and thus is compatible with either the
	'-B' or '-local' options.

	This also does not undo the setting of the hostname or the timezone
	with either the command line ('-H <hostname>' and '-T <timezone>')
	or configuration file settings.

 -R	Specify '-R' to reset.  The cleanup logic is the same as with
	-X.  Unlike -X, with -R, processing stops after the cleanup is
	done.

 -M	Specify '-M' to allow multiple runs. This is useful for running
 	a series of tests. This option disables the safety feature that
	self-disables this script after it completes.
	
	to prevent accidentally running this
	script again, e.g. after real data has been migrated to this
	machine after using this script to bootstrap for production.

HELP OPTIONS:
 -h	Display short help message
 -man	Display man-style help message

EXAMPLES:
	=== FAST INSTALLATION (skipping package updates) ===

	su -
    	mkdir -p /hxdepots/reset
	cd /hxdepots/reset
	curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/main/src/reset_sdp.sh
	curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/main/src/r
	chmod +x reset_sdp.sh r
	./r

	Note that the 'r' wrapper script calls the reset_sdp.sh script with
	a pre-defined of flags optimized for fast operation.  The 'r' wrapper
	also handles log capture, writing to the file 'reset_sdp.log'.

	=== COMPREHENSIVE INSTALLATION ===

	su -
    	mkdir -p /hxdepots/reset
	cd /hxdepots/reset
	curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/main/src/reset_sdp.sh

	chmod +x reset_sdp.sh
	./reset_sdp.sh 2>&1 | tee reset_sdp.log

	=== CONFIGURED INSTALLATION ===

	su -
    	mkdir -p /hxdepots/reset
	cd /hxdepots/reset
	curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/main/src/reset_sdp.sh
	chmod +x reset_sdp.sh

 	### Generate a default config file:
	./reset_sdp.sh -C > settings.cfg

 	### Edit settings.cfg, changing the values as desired:
	vi settings.cfg

	./reset_sdp.sh -c settings.cfg 2>&1 | tee log.reset_sdp

	=== LOCAL INSTALL ===

	The following sample commands illustrate how to acquire the
	dependencies for running with '-local' on a machine that can reach
	the public internet.  The resulting file structure, with paths as
	shown, would need to be somehow copied to the machine where the
	this reset_sdp.sh script is to be run.  This can be used to
	facilitate operation on a machine over an "air gap" network.

	$ mkdir -p /hxdepots/helix_binaries
	$ cd /hxdepots/helix_binaries
	$ curl -k -s -O https://ftp.perforce.com/perforce/r21.1/bin.linux26x86_64/p4
	$ curl -k -s -O https://ftp.perforce.com/perforce/r21.1/bin.linux26x86_64/p4d
	$ curl -k -s -O https://ftp.perforce.com/perforce/r21.1/bin.linux26x86_64/p4broker
	$ curl -k -s -O https://ftp.perforce.com/perforce/r21.1/bin.linux26x86_64/p4p

	$ mkdir /hxdepots/downloads
	$ cd /hxdepots/downloads
	$ curl -k -s -O https://ftp.perforce.com/perforce/tools/
	$ curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/downloads/helix_installer.tgz
	$ tar -xzf helix_installer.tgz
	$ rsync -a hi/src /hxdepots/reset

	=== SDP DEV BRANCH TESTING ===

	The Helix Installer can be used to test SDP changes shelved to the SDP
	dev branch in The Workshop.  The following example illustrates testing
	a change in shelved changelist 23123:

	su -
    	mkdir -p /hxdepots/reset
	cd /hxdepots/reset
	curl -k -s -O https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/download/main/src/reset_sdp.sh

	./reset_sdp.sh -b dev,@23123 2>&1 | tee log.reset_sdp.CL23123

	After the first test, an iterative test cycle may follow on the same
	shelved changelist. For each test iteration, the shelved changelist
	is first updated in the workspace from which the change was originally
	shelved, e.g. with a command like 'p4 shelve -f -c 23123'.

	Then a new test can be done by calling reset_sdp.sh with the same
	arguments. The script will re-install the SDP cleanly, and then
	re-apply the updated shelved changelist.

	=== SDP TEST SUITE SUPPORT ===

	The Helix Installer can install the SDP in the Docker-based SDP
	Test Suite.  In that environment, the directory /sdp appears on
	the test VMs, shared from the host machine.  To deploy that SDP,
	use the '-d <sdp_dir>' flag, something like this:

	./reset_sdp.sh -d /sdp -fast 2>&1 | tee log.reset_sdp.test