mkdirs.sh.man.txt #18

USAGE for mkdirs.sh v7.2.9:

mkdirs.sh <instance> [-r <P4BinRel>] [-s <ServerID>] [-t <ServerType>] [-tp <TargetPort>] [-lp <ListenPort>] [-I <svc>[,<svc2>]] [-MDD /bigdisk] [-MCD /ckps] [-MLG /jnl] [-MDB1 /db1] [-MDB2 /db2] [-f] [-p] [-no_init|-no_systemd|-no_enable] [-fs|-ls] [-no_cron] [-no_firewall] [-test [-clean]] [-n] [-L <log>] [-d|-D]

OR

mkdirs.sh <instance> [-c <CfgFile>] [-f] [-p] [-no_init|-no_systemd|-no_enable] [-fs|-ls] [-no_cron] [-no_firewall] [-test [-clean]] [-n] [-L <log>] [-d|-D]

or

mkdirs.sh [-h|-man]


DESCRIPTION:

== Overview ==

This script initializes an SDP instance on a single machine.

This script is intended to support two scenarios:

* First time SDP installation on a given machine.  In this case, the
  user calls the install_sdp.sh script, which in turn calls this script.
  See 'install_sdp.sh -man' for more information.

* Adding new SDP instances (separate Helix Core data sets) to an existing
  SDP installation on a given machine. For this scenario, this mkdirs.sh
  script is called directly.

An SDP instance is a single Helix Core data set, with its own unique
set of one set of users, changelist numbers, jobs, labels, versioned
files, etc. An organization may run a single instance or multiple
instances.

This is intended to be run either as root or as the operating system
user account (OSUSER) that p4d is configured to run as, typically
'perforce'.  It should be run as root for the initial install.
Subsequent additions of new instances do not require root.

== Directory Structure ==

If an initial install as done by a user other than root, various
directories must exist and be writable and owned by 'perforce' before starting:

* /p4
* /hxcheckpoints
* /hxdepots
* /hxlogs
* /hxmetadata
* /hxmetadata2
* /opt/perforce/helix-sdp (optional; used for package installations)

The directories starting with '/hx' are configurable, and can be changed by
settings in the mkdirs.cfg file (or mkdirs.N.cfg), or with command line
options as illustrated here:

 -MDD /bigdisk
 -MCD /ckps
 -MLG /jnl
 -MDB1 /db1
 -MDB2 /db2

This script creates an init script in the /p4/N/bin directory.

== Crontab ==

Crontabs are generated for all server types.

After running this script, set up the crontab based on templates
generated as /p4/common/etc/cron.d.  For convenience, a sample crontab
is generated for the current machine as /p4/p4.crontab.<SDPInstance> 
(or /p4/p4.crontab.<SDPInstance>.new if the former name exists).

These files should be copied or merged into any existing files named
with this convention:

/p4/common/etc/cron.d/crontab.<osuser>.<host>

where <osuser> is the user that services run as (typically 'perforce'),
and <host> is the short hostname (as returned by a 'hostname -s' command).

== Init Mechanism ==

If this script is run as root, the init mechanism (Systemd or SysV) is
configured for installed services.

The Systemd mechanim is used if the the /etc/systemd/system folder exists and
systemctl is in the PATH of the root user. Otherwise, the SysV init mechanism
is used.

== Firewall Configuration ==

This script checks to see if a known firewall type is available.  The
firewalld is checked using the command 'firewall-cmd --state' command,
and the ufw firewall is checked using the 'ufw status'.  If either firewall
is detected, the ports required for Helix Core applications installed are
opened in the firewall.  For more information, see the templates in these folders:

/p4/common/etc/firewalld
/p4/common/etc/ufw

If the firewall service is not online, no firewall coniguration is performed.

== SELinux Configuration ==

If Systemd is used and the semanage and restorecon utilities are available in
the PATH of the root user, then SELinux configuration for the installed
services is done.

REQUIRED PARAMETERS:
 <instance>
	Specify the SDP instance name to add.  This is a reference to the Perforce
	Helix Core data set.

OPTIONS:
 -s <ServerID>
	Specify the ServerID, overriding the REPLICA_ID setting in the configuration
	file.

 -S <TargetServerID>
	Specify the ServerID of the P4TARGET of the server being installed.
	Use this only when setting up an HA replica of an edge server.

 -t <ServerType>
	Specify the server type, overriding the SERVER_TYPE setting in the config
	file.  Valid values are:
	* p4d_commit - A master/commit server.
	* p4d_master - A synonym for p4d_commit.
	* p4d_replica - A replica with all metadata from the master (not
	  filtered in any way).
	* p4d_filtered_replica - A filtered replica or filtered forwarding
	  replica.
	* p4d_edge - An edge server.
	* p4d_edge_replica - Replica of an edge server. If used,
	  '-S <TargetServerID>' is required.
	* p4broker - An SDP host running only a standalone p4broker, with no p4d.
	* p4p - An SDP host running only a standalone p4p, with no p4d.
	* p4proxy - A synonym for p4p.

 -tp <TargetPort>
	Specify the target port. Use only if ServerType is p4p and p4broker.

 -lp <ListenPort>
	Specify the listen port. Use only if ServerType is p4p and p4broker.

 -I [<svc>[,<svc2>]]
	Specify additional init scripts to be added to /p4/<instance>/bin
	for the instance.

	By default, the p4p service is installed only if '-t p4proxy' is
	specified.  p4dtg is never installed by default.  Valid values
	to specify are 'p4p' and 'dtg' (for the P4DTG init script).

	If services are not installed by default, they can be added later
	using templates in /p4/common/etc/init.d. Also, templates for
	systemd service files that call the init scripts are supplied in
	/p4/common/etc/systemd/system.

 -MDD /bigdisk
 -MCD /ckps
 -MLG /jnl
 -MDB1 /db1
 -MDB2 /db2
	Specify the '-M*' optons to specify mount points, overriding
	DD/CD/LG/DB1/DB2 settings in the config file.  Sample:

	-MDD /bigdisk -MLG /jnl -MDB1 /fast

	If -MDB2 is not specified, it is set the the same value as -MDB1 if
	that is set, or else it defaults to the same default value as DB1.

 -c <CfgFile>
	Specify the path to the configuration file to use, overriding the
	default logic of finding the file based on naming convention.

 -f	Specify -f 'fast mode' to skip chown/chmod commands on depot files.
	This should only be used when you are certain the ownership and
	permissions are correct, and if you have large amounts of existing
	data for which the chown/chmod of the directory tree would be
	time-consuming and unnecessary.

 -p	Specify '-p' to halt processing after preflight checks are complete,
	and before actual processing starts. By default, processing starts
	immediately upon successful completion of preflight checks.

 -no_init
	Specify '-no_init' to avoid any service configuration, which
	is done by default if running as root (using systemd if available,
	otherwise SysV).  If '-no_init' is used, then neither systemd nor
	SysV init mechanism is configured for installed services.

	This option is implied if not running as root.

	This option is implied if '-test' is used.

 -no_systemd
	Specify '-no_systemd' to avoid using systemd, even if it
	appears to be available. By default, systemd is used if it
	appears to be available.

	This is helpful in operating in containerized test environments
	where systemd does not work even if it appears to be available.

	This option is implied if the systemctl command is not available
	in the PATH of the root user.

	This option is implied if '-no_init' is used.

 -no_enable
	Specify '-no_enable' to avoid enabling systemd services to start
	automatically after a reboot. If this option is used, systemd
	services will still be created, allowing services to be manually
	started and stopped.

	Specifically, this options means the 'systemctl enable' command
	is not run for generated services.

 -no_cron
 	Specify '-no_cron' to avoid loading the crontab.

	A crontab file is generated in the /p4 directory, but
	but with '-no_cron, this file is not loaded as the active crontab.

 -no_firewall
 	Specify '-no_firewall' to avoid attempting firewall configuration.

	By default, if the firewalld service is found to be running,
	it is configured so that the ports for p4d and p4broker are
	open.

 -fs	Specify '-full' when calling gen_sudoers.sh to install a new, full
	sudoers file. This option is only available if running as root.

	This option is mutually exclusive with '-ls'.

	See 'gen_sudoers.sh -man' for more info.

 -ls	Specify '-limited' when calling gen_sudoers.sh to install a new,
	limited sudoers file. This option is only available if running as
	root.

	This option is mutually exclusive with '-fs'.

	See 'gen_sudoers.sh -man' for more info.

 -L <log>
	Specify the path to a log file, or the special value 'off' to disable
	logging.  By default, all output (stdout and stderr) goes to this file
	in the current directory:

	mkdirs.<instance>.<datestamp>.log

	NOTE: This script is self-logging.  That is, output displayed on the
	screen is simultaneously captured in the log file.  Do not run this
	script with redirection operators like '> log' or '2>&1', and do not
	use 'tee'.

DEBUGGING OPTIONS:
 -test
 	Specify '-test' to execute a simulated install to /tmp/p4 as the install
	root (rather than /p4), and with the mount point directories specified in
	the configuration file prefixed with /tmp/hxmounts, defaulting to:
	* /tmp/hxmounts/hxdepots
	* /tmp/hxmounts/hxlogs
	* /tmp/hxmounts/hxmetadata

	This option implies '-no_init'.
 
 -clean
	Specify '-clean' with '-test' to clean up from prior test installs,
	which will result in removal of files/folders installed under /tmp/hxmounts
	and /tmp/p4.

 	Do not specify '-clean' if you want to test a series of installs.

 -n	No-Op.  In No-Op mode, no actions that affect data or structures are
 	taken.  Instead, commands that would be run are displayed.  This is
	an alternative to -test. Unlike '-p' which stops after the preflight
	checks, with '-n' more processing logic can be exercised, with greater
	detail about what commands that would be executed without '-n'.

 -d     Increase verbosity for debugging.

 -D     Set extreme debugging verbosity, using bash '-x' mode. Also implies -d.

HELP OPTIONS:
 -h	Display short help message
 -man	Display man-style help message

FILES:
	The mkdirs.sh script uses a configuration file for many settings.  A
	sample file, mkdirs.cfg, is included with the SDP.  After determining
	your SDP instance name (e.g. '1' or 'abc'), create a configuration
	file for it named mkdirs.<N>.cfg, replacing 'N' with your instance.

	Running 'mkdirs.sh N' will load configuration settings from mkdirs.N.cfg.

UPGRADING SDP:
	This script can be useful in testing and upgrading to new versions of
	the SDP, when the '-test' flag is used.

EXAMPLES:
	Example 1: Setup of first instance

	Setup of the first instance on a machine using the default instance name,
	'1', executed after using sudo to become root:

	$ sudo su -
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ vi mkdirs.cfg

	Adjust settings as desired, e.g P4PORT, P4BROKERPORT, etc.

	$ ./mkdirs.sh 1

	A log will be generated, mkdirs.1.<timestamp>.log

	Example 2: Setup of additional instance named 'abc'.

	Setup a second instance on the machine, which will be a separate Helix
	Core instance with its own P4ROOT, its own set of users and
	changelists, and its own license file (copied from the master instance).

	Note that while the first run of mkdirs.sh on a given machine should be
	done as root, but subsequent instance additions can be done as the
	'perforce' user (or whatever operating system user accounts Perforce
	Helix services run as).

	$ sudo su - perforce
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ cp mkdirs.cfg mkdirs.abc.cfg
	$ chmod +w mkdirs.abc.cfg
	$ vi mkdirs.abc.cfg

	Adjust settings in mkdirs.abc.cfg as desired, e.g P4PORT, P4BROKERPORT, etc.

	$ ./mkdirs.sh abc

	A log will be generated, mkdirs.abc.<timestamp>.log

	Example 3: Setup of additional instance named 'alpha' to run a standalone p4p
	targeting commit.example.com:1666 and listening locally on port 1666.

	$ sudo su -
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ ./mkdirs.sh alpha -t p4p -tp commit.example.com:1666 -lp 1666

	Example 4: Setup of instance named '1' to run a standalone p4broker
	targeting commit.example.com:1666 and listening locally on port 1666.

	$ sudo su -
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ ./mkdirs.sh 1 -t p4broker -tp commit.example.com:1666 -lp 1666

	Example 5: Setup 2 instances A and B with limited sudoers on a fresh new machine:

	$ sudo su -
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ cp mkdirs.cfg mkdirs.A.cfg

	Adjust settings in mkdirs.A.cfg as desired, e.g P4PORT, P4BROKERPORT, etc.

	$ cp mkdirs.A.cfg mkdirs.B.cfg

	Adjust settings in mkdirs.B.cfg as desired, e.g P4PORT, P4BROKERPORT, etc.
	Ensure port numbers do not conflict. Then generate Instance A:

	$ ./mkdirs.sh A -ls

	A log will be generated, mkdirs.A.<timestamp>.log

	Next generate instnace B, updating the limited sudoers to reference both
	instances.

	$ ./mkdirs.sh B -ls

SEE ALSO:
	See 'install_sdp.sh -man' for more info on installing on a new machine.

	See 'gen_sudoers.sh -man' for more info on generating/replacing sudoers.

	See template:
	* systemd service file templates: /p4/common/etc/systemd/system
	* firewalld templates: /p4/common/etc/firewalld
	* ufw firewall templates: /p4/common/etc/ufw
	* Init script templates: /p4/common/etc/init.d