USAGE for mkdirs.sh v4.2.0: mkdirs.sh <instance> [-s <ServerID>] [-t <server_type>] [-MDD /bigdisk] [-MLG /jnl] [-MDB1 /db1] [-MDB2 /db2 ] [-f] [-p] [-test [-clean]] [-n] [-L <log>] [-d|-D] or mkdirs.sh [-h|-man] DESCRIPTION: This script initializes an SDP instance on a single machine. This script is intended to support two scenarios: * First time SDP installation on a given machine. * Adding new SDP instances (separate Helix Core data sets) to an existing SDP installation on a given machine. And SDP instance is a single Helix Core data set, with its own unique set of one set of users, changelist numbers, jobs, labels, versioned files, etc. An organization may run a single instance or multiple instances. This is intended to be run either as root or as the operating system user account (OSUSER) that p4d is configured to run as, typically 'perforce'. It should be run as root for the initial install. Subsequent additions of new instances do not require root. If an initial install as done by a user other than root, various directories must exist and be writable and owned by 'perforce' before starting: * /p4 * /hxdepots * /hxlogs * /hxmetadata This script creates an init script in the /p4/N/bin directory. After running this script, set up the crontab based on templates generated in /p4/common/etc/cron.d. For convenience, a sample cronat is generated for the current machine in /p4/common/etc/cron.d named crontab.<osuser>.<host> where <osuser> is the user that services run as (typically 'perforce'), and <host> is the short hostname (as returned by a 'hostname -s' command). Next, put the license file in place in the P4ROOT dir, and launch the server with the init script. Then run /p4/common/bin/p4master_run instance /p4/common/bin/live_checkpoint.sh and then run both the daily_checkpoint.sh and recreate_db_checkpoint.sh to make sure everything is working before setting up the crontab. Also run /p4/common/bin/p4master_run <instance> /p4/common/bin/p4review.py <instance> to make sure the review script is working properly. If you intend to use Swarm, you can skip configuration of the review daemon, and instead configure Swarm to handle review-style email notifications. REQUIRED PARAMETERS: <instance> Specify the SDP instance name to add. This is a reference to the Perforce Helix Core data set. OPTIONS: -s <ServerID> Specify the ServerID, overriding the REPLICA_ID setting in the configuration file. -S <TargetServerID> Specify the ServerID of the P4TARGET of the server being installed. Use this when setting up an edge server. -t <server_type> Specify the server type, overriding the SERVER_TYPE setting in the config file. Valid values are: * p4d_master - A master/commit server. * p4d_replica - A replica with all metadata from the master (not filtered in any way). * p4d_filtered_replica - A filtered replica or filtered forwarding replica. * p4d_edge - An edge server. * p4d_edge_replica - Replica of an edge server. If used, '-S <TargetServerID>' is required. * p4broker - An SDP host running only a broker, with no p4d. * p4proxy - An SDP host running a proxy (maybe with a broker in front), with no p4d. -MDD /bigdisk -MLG /jnl -MDB1 /db1 -MDB2 /db2 Specify the '-M*' to specify mount points, overriding DD/LG/DB1/DB2 settings in the config file. Sample: -MDD /bigdisk -MLG /jnl -MDB1 /fast If -MDB2 is not specified, it is set the the same value as -MDB1 if that is set, or else it defaults to the same default value as DB1. -f Specify -f 'fast mode' to skip chown/chmod commands on depot files. This should only be used when you are certain the ownership and permissions are correct, and if you have large amounts of existing data for which the chown/chmod of the directory tree would be slow. -p Specify '-p' to halt processing after preflight checks are complete, and before actual processing starts. By default, procesing starts immediately upon successful completion of preflight checks. -L <log> Specify the path to a log file, or the special value 'off' to disable logging. By default, all output (stdout and stderr) goes to this file in the current directory: mkdirs.<instance>.<datestamp>.log NOTE: This script is self-logging. That is, output displayed on the screen is simultaneously captured in the log file. Do not run this script with redirection operators like '> log' or '2>&1', and do not use 'tee.' DEBUGGING OPTIONS: -test Specify '-test' to execute a simulated install to /tmp/p4 as the install root (rather than /p4), and with the mount point directories specifed in the configuration file prefixed with /tmp/hxmounts, defaulting to: * /tmp/hxmounts/hxdepots * /tmp/hxmounts/hxlogs * /tmp/hxmounts/hxmetadata -clean Specify '-clean' with '-test' to clean up from prior test installs, which will result in removal of files/folders installed under /tmp/hxmounts and /tmp/p4. Do not specify '-clean' if you want to test a series of installs. -n No-Op. In No-Op mode, no actions that affect data or structures are taken. Instead, commands that would be run are displayed. This is an alternative to -test. Unlike '-p' which stops after the preflight checks, with '-n' more processing logic can be exercised, with greater detail about what commands that would be executed without '-n'. -d Increase verbosity for debugging. -D Set extreme debugging verbosity, using bash '-x' mode. Also implies -d. HELP OPTIONS: -h Display short help message -man Display man-style help message FILES: The mkdirs.sh script uses a configuration file for many settings. A sample file, mkdirs.cfg, is included with the SDP. After determining your SDP instance name (e.g. '1' or 'abc'), create a configuration file for it named mkdirs.<N>.cfg, replacing 'N' with your instance. Running 'mkdirs.sh N' will load configuration settings from mkdirs.N.cfg. UPGRADING SDP: This script can be useful in testing and upgrading to new versions of the SDP, when the '-test' flag is used. EXAMPLES: Example 1: Setup of first instance Setup of the first instance on a machine using the default instance name, '1', executed after using sudo to become root: $ sudo su - $ cd /hxdepots/sdp/Server/Unix/setup $ vi mkdirs.cfg # Adjust settings as desired, e.g P4PORT, P4BROKERPORT, etc. $ ./mkdirs.sh 1 A log will be generated, mkdirs.1.<timestamp>.log Example 2: Setup of additional instance named 'abc'. Setup a second instanced on the machine, which will be a seprate Helix Core instantce with its own P4ROOT, its own set of users and changelists, and its own license file (copied from the master instance). Note that while the first run of mkdirs.sh on a given machine should be done as root, but subsequent instane additions should be done as the 'perforce' user (or whatever operating system user accounts Perforce Helix services run as). $ sudo su - perforce $ cd /hxdepots/sdp/Server/Unix/setup $ cp -p mkdirs.cfg mkdirs.abc.cfg $ vi mkdirs.abc.cfg # Adjust settings in mkdirs.abc.cfg as desired, e.g P4PORT, P4BROKERPORT, etc. $ ./mkdirs.sh abc A log will be generated, mkdirs.abc.<timestamp>.log
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#25 | 30912 | C. Thomas Tyler | Updated generated script man pages. | ||
#24 | 30782 | C. Thomas Tyler |
Added new install_sdp.sh script and supporting documentation. The new install_sdp.sh makes SDP independent of the separate Helix Installer software (the reset_sdp.sh script). The new script greatly improves the installation experience for new server machines. It is ground up rewrite of the reset_sdp.sh script. The new script preserves the desired behaviors of the original Helix Installer script, but is focused on the use case of a fresh install on a new server machine. With this focus, the scripts does not have any "reset" logic, making it completely safe. Added various files and functionalityfrom Helix Installer into SDP. * Added firewalld templates to SDP, and added ufw support. * Improved sudoers generation. * Added bash shell templates. This script also installs in the coming SDP Package structure. New installs use a modified SDP structure that makes it so the /p4/sdp and /p4/common now point to folders on the local OS volume rather than the /hxepots volume. The /hxdepots volume, which is often NFS mounted, is still used for depots and checkpoints, and for backups. The new structure uses a new /opt/perforce/helix-sdp structure under which /p4/sdp and /p4/common point. This structure also contains the expaneded SDP tarball, downloads, helix_binaries, etc. This change represents the first of 3-phase rollout of the new package structure. In this first phase, the "silent beta" phase, the new structure is used for new installations only. This phase requires no changes to released SDP scripts except for mkdirs.sh, and even that script remains backward-compatible with the old structure if used independently of install_sdp.sh. If used with install_sdp.sh, the new structure is used. In the second phase (targeted for SPD 2024.2 release), the sdp_upgrade.sh script will convert existing installations to the new structure. In the third phase (targeted for SDP 2025.x), this script will be incorporated into OS pacakge installations for the helix-sdp package. Perforce internal wikis have more detail on this change. #review-30783 |
||
#23 | 30681 | C. Thomas Tyler |
Added gen_sudoers.sh script to generate a sudoers file for perforce OSUSER. This generates a more secure limited sudoers file. Previously, adding a sudoers entry for the OSUSER (usually 'perforce') was done only by the Helix Installer. In the Helix Installer variant, a single "one-size-filts-all" sudoers file was used, with the following characteristics: * The instances for Helix Core services were referenced with a '*' wildcard to match all SDP instances, which has since been determined to introduce a vulnerability. In this new script, the wildcard is replaced with separate entries for each SDP instance. * There were entries for all known paths of utilities like lslocks, setcap, and getcap. This new script generates the correct path valid for the current machine. With this change, the functionality will be available in the SDP directly. This new gen_sudoers.sh script can be called by mkdirs.sh directly to update the sudoers file each time a new SDP instance is added, if the new '-fs' (full sudo) or '-ls' (limited sudoers) entries are used. There is no change to the default behavior of mkdirs.sh; only a change if new options are utilized. This script comes with docs and examples for the new script as well as doc changes for mkdirs.sh. (Also added missing documentation for the '-no_enable' option). Further changes needed: * Add doc reference in SDP_Guide.Unix.adoc |
||
#22 | 30384 | C. Thomas Tyler | Updated generated script man pages. | ||
#21 | 30039 | C. Thomas Tyler | Updated generated script man pages. | ||
#20 | 30030 | C. Thomas Tyler | Updated generated script man pages. | ||
#19 | 29699 | C. Thomas Tyler | Updated generated script man pages. | ||
#18 | 29621 | C. Thomas Tyler | Updated generated script man pages. | ||
#17 | 29399 | C. Thomas Tyler | Updated generated script man pages. | ||
#16 | 29203 | C. Thomas Tyler | Updated generated script man pages. | ||
#15 | 28410 | C. Thomas Tyler | Updated generated script man pages. | ||
#14 | 28216 | C. Thomas Tyler | Updated generated script man pages. | ||
#13 | 28092 | C. Thomas Tyler | Updated generated script man pages. | ||
#12 | 27899 | C. Thomas Tyler | Updated generated script man pages. | ||
#11 | 27820 | C. Thomas Tyler | Updated generated script man pages. | ||
#10 | 27759 | C. Thomas Tyler | Updated generated script man pages. | ||
#9 | 27724 | C. Thomas Tyler | Updated generated script man pages. | ||
#8 | 27396 | C. Thomas Tyler | Updated generated script man pages. | ||
#7 | 27317 | C. Thomas Tyler | Updated generated script man pages. | ||
#6 | 27252 | C. Thomas Tyler | Updated generated script man pages. | ||
#5 | 27155 | C. Thomas Tyler | Updated generated script man pages. | ||
#4 | 27073 | C. Thomas Tyler | Updated generated script man pages. | ||
#3 | 27032 | C. Thomas Tyler | Updated generated script man pages. | ||
#2 | 27009 | C. Thomas Tyler | chmod +w | ||
#1 | 27006 | C. Thomas Tyler | Updated generated docs. |