mkdirs.sh.man.txt #11

  • //
  • guest/
  • perforce_software/
  • sdp/
  • dev/
  • doc/
  • gen/
  • mkdirs.sh.man.txt
  • View
  • Commits
  • Open Download .zip Download (8 KB)
USAGE for mkdirs.sh v4.6.1:

mkdirs.sh <instance> [-s <ServerID>] [-t <server_type>] [-I <svc>[,<svc2>]] [-MDD /bigdisk] [-MLG /jnl] [-MDB1 /db1] [-MDB2 /db2] [-f] [-p] [-test [-clean]] [-n] [-L <log>] [-d|-D]

or

mkdirs.sh [-h|-man]


DESCRIPTION:

This script initializes an SDP instance on a single machine.

This script is intended to support two scenarios:

* First time SDP installation on a given machine.
* Adding new SDP instances (separate Helix Core data sets) to an existing
  SDP installation on a given machine.

And SDP instance is a single Helix Core data set, with its own unique
set of one set of users, changelist numbers, jobs, labels, versioned
files, etc. An organization may run a single instance or multiple
instances.

This is intended to be run either as root or as the operating system
user account (OSUSER) that p4d is configured to run as, typically
'perforce'.  It should be run as root for the initial install.
Subsequent additions of new instances do not require root.

If an initial install as done by a user other than root, various
directories must exist and be writable and owned by 'perforce' before starting:

* /p4
* /hxdepots
* /hxlogs
* /hxmetadata

This script creates an init script in the /p4/N/bin directory.

After running this script, set up the crontab based on templates
generated in /p4/common/etc/cron.d.  For convenience, a sample cronat
is generated for the current machine in /p4/common/etc/cron.d named

crontab.<osuser>.<host>

where <osuser> is the user that services run as (typically 'perforce'),
and <host> is the short hostname (as returned by a 'hostname -s' command).


Next, put the license file in place in the P4ROOT dir, and launch the server
with the init script.

Then run /p4/common/bin/p4master_run instance /p4/common/bin/live_checkpoint.sh
and then run both the daily_checkpoint.sh and recreate_db_checkpoint.sh to
make sure everything is working before setting up the crontab.

Also run /p4/common/bin/p4master_run <instance> /p4/common/bin/p4review.py <instance>
to make sure the review script is working properly.  If you intend to use
Swarm, you can skip configuration of the review daemon, and instead configure
Swarm to handle review-style email notifications.

REQUIRED PARAMETERS:
 <instance>
	Specify the SDP instance name to add.  This is a reference to the Perforce
	Helix Core data set.

OPTIONS:
 -s <ServerID>
	Specify the ServerID, overriding the REPLICA_ID setting in the configuration
	file.

 -S <TargetServerID>
	Specify the ServerID of the P4TARGET of the server being installed.
	Use this only when setting up an HA replica of an edge server.

 -t <server_type>
	Specify the server type, overriding the SERVER_TYPE setting in the config
	file.  Valid values are:
	* p4d_master - A master/commit server.
	* p4d_replica - A replica with all metadata from the master (not
	  filtered in any way).
	* p4d_filtered_replica - A filtered replica or filtered forwarding
	  replica.
	* p4d_edge - An edge server.
	* p4d_edge_replica - Replica of an edge server. If used,
	  '-S <TargetServerID>' is required.
	* p4broker - An SDP host running only a broker, with no p4d.
	* p4proxy - An SDP host running a proxy (maybe with a broker in front),
	  with no p4d.

 -I [<svc>[,<svc2>]]
	Specify additional init scripts to be added to /p4/<instance>/bin
	for the instance.

	By default, the p4p service is installed only if '-t p4proxy' is
	specified, and p4dtg is never installed by default.  Valid values
	to specify are 'p4p' and 'dtg' (for the P4DTG init script).

	If services are not installed by default, they can be added later
	using templates in /p4/common/etc/init.d. Also, templates for
	systemd service files are supplied in /p4/common/etc/systemd/system.

 -MDD /bigdisk
 -MLG /jnl
 -MDB1 /db1
 -MDB2 /db2
	Specify the '-M*' to specify mount points, overriding DD/LG/DB1/DB2
	settings in the config file.  Sample:

	-MDD /bigdisk -MLG /jnl -MDB1 /fast

	If -MDB2 is not specified, it is set the the same value as -MDB1 if
	that is set, or else it defaults to the same default value as DB1.

 -f	Specify -f 'fast mode' to skip chown/chmod commands on depot files.
	This should only be used when you are certain the ownership and
	permissions are correct, and if you have large amounts of existing
	data for which the chown/chmod of the directory tree would be
	slow.

 -p	Specify '-p' to halt processing after preflight checks are complete,
	and before actual processing starts. By default, processing starts
	immediately upon successful completion of preflight checks.

 -L <log>
	Specify the path to a log file, or the special value 'off' to disable
	logging.  By default, all output (stdout and stderr) goes to this file
	in the current directory:

	mkdirs.<instance>.<datestamp>.log

	NOTE: This script is self-logging.  That is, output displayed on the
	screen is simultaneously captured in the log file.  Do not run this
	script with redirection operators like '> log' or '2>&1', and do not
	use 'tee'.

DEBUGGING OPTIONS:
 -test
 	Specify '-test' to execute a simulated install to /tmp/p4 as the install
	root (rather than /p4), and with the mount point directories specified in
	the configuration file prefixed with /tmp/hxmounts, defaulting to:
	* /tmp/hxmounts/hxdepots
	* /tmp/hxmounts/hxlogs
	* /tmp/hxmounts/hxmetadata
 
 -clean
	Specify '-clean' with '-test' to clean up from prior test installs,
	which will result in removal of files/folders installed under /tmp/hxmounts
	and /tmp/p4.

 	Do not specify '-clean' if you want to test a series of installs.

 -n	No-Op.  In No-Op mode, no actions that affect data or structures are
 	taken.  Instead, commands that would be run are displayed.  This is
	an alternative to -test. Unlike '-p' which stops after the preflight
	checks, with '-n' more processing logic can be exercised, with greater
	detail about what commands that would be executed without '-n'.

 -d     Increase verbosity for debugging.

 -D     Set extreme debugging verbosity, using bash '-x' mode. Also implies -d.

HELP OPTIONS:
 -h	Display short help message
 -man	Display man-style help message

FILES:
	The mkdirs.sh script uses a configuration file for many settings.  A
	sample file, mkdirs.cfg, is included with the SDP.  After determining
	your SDP instance name (e.g. '1' or 'abc'), create a configuration
	file for it named mkdirs.<N>.cfg, replacing 'N' with your instance.

	Running 'mkdirs.sh N' will load configuration settings from mkdirs.N.cfg.

UPGRADING SDP:
	This script can be useful in testing and upgrading to new versions of
	the SDP, when the '-test' flag is used.

EXAMPLES:
	Example 1: Setup of first instance

	Setup of the first instance on a machine using the default instance name,
	'1', executed after using sudo to become root:
	$ sudo su -
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ vi mkdirs.cfg

	# Adjust settings as desired, e.g P4PORT, P4BROKERPORT, etc.

	$ ./mkdirs.sh 1

	A log will be generated, mkdirs.1.<timestamp>.log

	Example 2: Setup of additional instance named 'abc'.

	Setup a second instance on the machine, which will be a separate Helix
	Core instance with its own P4ROOT, its own set of users and
	changelists, and its own license file (copied from the master instance).

	Note that while the first run of mkdirs.sh on a given machine should be
	done as root, but subsequent instance additions should be done as the
	'perforce' user (or whatever operating system user accounts Perforce
	Helix services run as).

	$ sudo su - perforce
	$ cd /hxdepots/sdp/Server/Unix/setup
	$ cp -p mkdirs.cfg mkdirs.abc.cfg
	$ vi mkdirs.abc.cfg

	# Adjust settings in mkdirs.abc.cfg as desired, e.g P4PORT, P4BROKERPORT, etc.

	$ ./mkdirs.sh abc

	A log will be generated, mkdirs.abc.<timestamp>.log

	Example 3: Setup of additional instance named 'alpha' to run a p4p:

	$ ./mkdirs.sh alpha -t p4proxy

# Change User Description Committed
#29 31557 C. Thomas Tyler Updated generated script man pages.
#28 31366 C. Thomas Tyler Updated generated script man pages.
#27 31187 Will Kreitzmann Updated generated script man pages.
#26 31064 C. Thomas Tyler Updated generated script man pages.
#25 30912 C. Thomas Tyler Updated generated script man pages.
#24 30782 C. Thomas Tyler Added new install_sdp.sh script and supporting documentation.

The new install_sdp.sh makes SDP independent of the separate
Helix Installer software (the reset_sdp.sh script).  The new
script greatly improves the installation experience for new
server machines. It is ground up rewrite of the reset_sdp.sh
script. The new script preserves the desired behaviors of the
original Helix Installer script, but is focused on the use
case of a fresh install on a new server machine. With this focus,
the scripts does not have any "reset" logic, making it completely
safe.

Added various files and functionalityfrom Helix Installer into SDP.
* Added firewalld templates to SDP, and added ufw support.
* Improved sudoers generation.
* Added bash shell templates.

This script also installs in the coming SDP Package structure.
New installs use a modified SDP structure that makes it so the
/p4/sdp and /p4/common now point to folders on the local OS
volume rather than the /hxepots volume. The /hxdepots volume,
which is often NFS mounted, is still used for depots and
checkpoints, and for backups.

The new structure uses a new /opt/perforce/helix-sdp structure
under which /p4/sdp and /p4/common point. This structure also
contains the expaneded SDP tarball, downloads, helix_binaries,
etc.

This change represents the first of 3-phase rollout of the new
package structure. In this first phase, the "silent beta" phase,
the new structure is used for new installations only. This phase
requires no changes to released SDP scripts except for mkdirs.sh,
and even that script remains backward-compatible with the old
structure if used independently of install_sdp.sh.  If used with
install_sdp.sh, the new structure is used.

In the second phase (targeted for SPD 2024.2 release), the
sdp_upgrade.sh script will convert existing installations to
the new structure.

In the third phase (targeted for SDP 2025.x), this script will
be incorporated into OS pacakge installations for the helix-sdp
package.

Perforce internal wikis have more detail on this change.

#review-30783
#23 30681 C. Thomas Tyler Added gen_sudoers.sh script to generate a sudoers file for perforce OSUSER.

This generates a more secure limited sudoers file.

Previously, adding a sudoers entry for the OSUSER (usually 'perforce') was done only by the Helix Installer. In the Helix Installer variant, a single "one-size-filts-all" sudoers file was used, with the following characteristics:
* The instances for Helix Core services were referenced with a '*' wildcard to match all SDP instances, which has since been determined to introduce a vulnerability. In this new script, the wildcard is replaced with separate entries for each SDP instance.
* There were entries for all known paths of utilities like lslocks, setcap, and getcap. This new script generates the correct path valid for the current machine.

With this change, the functionality will be available in the SDP directly. This new gen_sudoers.sh script can be called by mkdirs.sh directly to update the sudoers file each time a new SDP instance is added, if the new '-fs' (full sudo) or '-ls' (limited sudoers) entries are used. There is no change to the default behavior of mkdirs.sh; only a change if new options are utilized.

This script comes with docs and examples for the new script as well as doc changes for mkdirs.sh. (Also added missing documentation for the '-no_enable' option).

Further changes needed:
* Add doc reference in SDP_Guide.Unix.adoc
#22 30384 C. Thomas Tyler Updated generated script man pages.
#21 30039 C. Thomas Tyler Updated generated script man pages.
#20 30030 C. Thomas Tyler Updated generated script man pages.
#19 29699 C. Thomas Tyler Updated generated script man pages.
#18 29621 C. Thomas Tyler Updated generated script man pages.
#17 29399 C. Thomas Tyler Updated generated script man pages.
#16 29203 C. Thomas Tyler Updated generated script man pages.
#15 28410 C. Thomas Tyler Updated generated script man pages.
#14 28216 C. Thomas Tyler Updated generated script man pages.
#13 28092 C. Thomas Tyler Updated generated script man pages.
#12 27899 C. Thomas Tyler Updated generated script man pages.
#11 27820 C. Thomas Tyler Updated generated script man pages.
#10 27759 C. Thomas Tyler Updated generated script man pages.
#9 27724 C. Thomas Tyler Updated generated script man pages.
#8 27396 C. Thomas Tyler Updated generated script man pages.
#7 27317 C. Thomas Tyler Updated generated script man pages.
#6 27252 C. Thomas Tyler Updated generated script man pages.
#5 27155 C. Thomas Tyler Updated generated script man pages.
#4 27073 C. Thomas Tyler Updated generated script man pages.
#3 27032 C. Thomas Tyler Updated generated script man pages.
#2 27009 C. Thomas Tyler chmod +w
#1 27006 C. Thomas Tyler Updated generated docs.