USAGE for mkrep.sh v3.3.0: mkrep.sh -t -s -r [-f ] [-os] [-p] [-N ] [-i ] [-L ] [-v] [-n] [-D] or mkrep.sh [-h|-man|-V] DESCRIPTION: This script simplifies the task of creating Helix Core replicas and edge servers, and helps ensure they are setup with best practices. This script executes as two phases. In Phase 1, this script does all the metadata configuration to be executed on the master server that must be baked into a seed checkpoint for creating the replica/edge. This essentially captures the planning for a new replica, and can be done before the physical infrastructure (e.g. hardware, storage, and networking) is ready. Phase 1, fully automated by this script, takes only seconds to run. In Phase 2, this script provides information for the manual steps needed to create, transfer, and load seed checkpoints onto the replica/edge. The guidance is specific to type of replica created, based on the command line flags provided to this script. This processing can take a while for large data sets, as it involves creating and transporting checkpoints. Before using this script, a set of geographic site tags must be defined. See the FILES: below for details on a site tags. This script adheres to the these SDP Standards: * Server Spec Naming Standard: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Guide.Unix.html#_server_spec_naming_standard * Journal Prefix Standard: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Guide.Unix.html#_the_journalprefix_standard In Phase 1, this script does the following to help create a replica or edge server: * Generates the server spec for the new replica. * Generates a server spec for master server (if needed). * Sets configurables ('p4 configure' settings) for replication. * Selects the correct 'Services' based on replica type. * Creates service user for the replica, and sets a password. * Creates service user for the master (if needed), and sets a password. * Adds newly created service user(s) to the group 'ServiceUsers'. * Verifies the group ServiceUsers is granted super access in the protections table (and with '-p', also updates Protections). After these steps are completed, in Phase 2, detailed instructions are presented to guide the user through the remaining steps needed to complete the deployment of the replica. This starts with creating a new checkpoint to capture all the metadata changes made by this script in Phase 1. SERVICE USERS: Service users created by this script are always of type 'service', and so will not consume a licensed seat. Service users also have an 'AuthMethod' of 'perforce' (not 'ldap') as is required by 'p4d' for 'service' users. Passwords set for service users are long 32 character random strings that are not stored, as they are never needed. Login tickets for service users are generated using: p4login -service -v OPTIONS: -t [N] Specify the replica type tag. The type corresponds to the 'Type:' and 'Services:' field of the server spec, which describes the type of services offered by a given replica. Valid type values are: * ha: High Availability standby replica, for 'p4 failover' (P4D 2018.2+) * ham: High Availability metadata-only standby replica, for 'p4 failover' (P4D 2018.2+) * ro: Read-Only standby replica. (Discouraged; Use 'ha' instead for 'p4 failover' support.) * rom: Read-Only standby replica, Metadata only. (Discouraged; Use 'ham' instead for 'p4 failover' support.) * fr: Forwarding Replica (Unfiltered). * fs: Forwarding Standby (Unfiltered). * frm: Forwarding Replica (Unfiltered, Metadata only). * fsm: Forwarding Standby (Unfiltered, Metadata only). * ffr: Filtered Forwarding Replica. Not a valid failover target. * edge: Edge Server. Filtered by definition. Replicas with 'standby' are always unfiltered, and use the 'journalcopy' method of replication, which copies a byte-for-byte verbatim journal file rather than one that is merely logically equivalent. The tag has several purposes: 1. Short Hand. Each tag represents a combination of 'Type:' and fully qualified 'Services:' values used in server specs. 2. Distillation. Only the most useful Type/Services combinations have a shorthand form 3. For forwarding replicas, the name includes the critical distinction of whether any replication filtering is used; as filtering of any kind disqualifies a replica from being a potential failover target. (No such distinction is needed for edge servers, which are filtered by definition). -s Specify a geographic site tag indicating the location and/or data center where the replica will physically be located. Valid site tags are defined in the site tags file: /p4/common/config/SiteTags.cfg A sample SiteTags.cfg file that is here: /p4/common/config/SiteTags.cfg.sample -r Specify the DNS name of the server machine on which the new replica will run. This is used in the 'ExternalAddress:' field of the replica's ServerID, and also used in instructions to the user for steps after metadata configuration is done by this script. -f Specify ServerID of the P4TARGET server from which we are replicating. This is used to populate the 'ReplicatingFrom' field of the server spec. The value must be a valid ServerID. This option should be used if the target is something other than the master. For example, to create an HA replica of an edge server, you might specify something like '-f p4d_edge_syd'. -os Specify the '-os' option to overwrite an exising server spec. By default, this script will abort of the server spec to be generated already exists on the Helix Core server. Specify this option to overwrite the existing server spec. -p This script always performs a check to ensure that the Protections table grants super access to the group ServiceUsers. By default, an error is displayed if the check fails, i.e. if super user access for the group ServiceUsers cannot be verified. This is because, by default, we want to avoid making changes to the Protections table. Some sites have local policies or custom automation that requires site-specific procedures to update the Protections table. If '-p' is specified, an attempt is made to append the Protections table an entry like: super group ServiceUsers * //... This option may not be suitable for use on servers that have custom automation managing the Protections table. -N Specify '-N ', where N is an integer. This is used to indicate that multiple replicas of the same type are to be created at the same site. The value specified with '-N' must be a numeric value. Left-padding with zeroes is allowed. For example, '-N 04' is allowed, and 'N A7' is not (as it is not numeric). This affects the ServerID to be generated. For example, the options '-t edge -s syd' would result in a ServerID of p4d_edge_syd. To create a second edge in the same site, use '-t edge -s syd -N 2' to generate p4d_edge2_syd. -i Specify the SDP Instance. If not specified and the SDP_INSTANCE environment is defined, that value is used. If SDP_INSTANCE is not defined, the '-i ' argument is required. -v Set verbosity 1-5 (-v1 = quiet, -v5 = highest). -L Specify the path to a log file, or the special value 'off' to disable logging. By default, all output (stdout and stderr) goes in the logs directory referenced by $LOGS environment variable, in a file named mkrep..log NOTE: This script is self-logging. That is, output displayed on the screen is simultaneously captured in the log file. Using redirection operators like '> log' or '2>&1' are not necessary, nor is using 'tee.' -n No-Op. Prints commands instead of running them. -D Set extreme debugging verbosity. HELP OPTIONS: -h Display short help message -man Display man-style help message -V Display version info for this script and its libraries. FILES: This Site Tags file defines the list of valid geographic site tags: /p4/common/config/SiteTags.cfg The contains one-line entries of the form: : where is a short alphanumeric tag name for a geographic location, data center, or other useful distinction. This tag is incorporated into the ServerID of replicas or edge servers created by this script. Tag names should be kept short, ideally no more than about 5 characters in length. The is a one-line text description of what the tag refers to, which may contain spaces and ASCII punctuation. Blank lines and lines starting with a '#' are considered comments and are ignored. REPLICA SERVER MACHINE SETUP: The replica/edge server machine must be have the SDP structure installed, either using the mkdirs.sh script included in the SDP, or the Helix Installer for 'green field' installations. When setting up an edge server, a replica of an edge server, or filtered replica, confirm that the JournaPrefix Standard (see URL above) structure has the separate checkpoints folder as identified in the 'Second Form' in the standard. A baseline SDP structure can typically be extended by running commands like like these samples (assuming a ServerID of p4d_edge_syd or p4d_ha_edge_syd): mkdir /hxdepots/p4/1/checkpoints.edge_syd cd /p4/1 ln -s /hxdepots/p4/1/checkpoints.edge_syd CUSTOM PRE- AND POST- OPERATION AUTOMATION HOOKS: This script can execute custom pre- and post- processing scripts. This can be useful to incorporate site-specific elements of replica setup. If the file /p4/common/site/mkrep/pre-mkrep.sh exists and is executable, it will be executed before mkrep.sh processing. If the file /p4/common/site/mkrep/post-mkrep.sh exists and is executable, it will be executed after mkrep.sh processing. Pre- and post- processing scripts are called with the same command line arguments passed to this mkrep.sh script. The pre- and post- processing scripts can use or ignore arguments as needed, though it is required to implement the '-n' flag to operate in preview mode, taking no actions that affect data (just as this script behaves). Pre- and post- processing scripts are expected to exit with a zero exit code to indicate success, and non-zero to indicate failure. The custom pre-processing script is executed after standard preflight checks complete successfully. If a custom pre-processing script indicates a failure, processing is aborted before standard mkrep.sh processing occurs. The post-processing custom script is executed after the standard mkrep.sh processing is successful. If a post-processing custom script is detected, the instructions that would be provided to the user in Phase 2 are not displayed, as it is expected that the custom post- processing will alter or handle these steps. Success or failure of pre- and post- processing scripts is reported in the log. These scripts do not require independent logging, as all standard and error output is captured in the log of this mkrep.sh script. TIP: Be sure to fully test custom scripts in a test environment before incorporating them into production systems. EXAMPLES: EXAMPLE 1 - Set up a High Availability (HA) Replica of the master. Add an HA replica to instance 1 to run on host bos-helix-02: mkrep.sh -i 1 -t ha -s bos -r bos-helix-02 EXAMPLE 2 - Add an Edge Server to the topology. Add an Edge server to instance acme to run on host syd-helix-04: mkrep.sh -i acme -t edge -s syd -r syd-helix-04 EXAMPLE 3 - Setup an HA replica of an edge server. Add a HA replica of the edge server to instance acme to run on host syd-helix-05: mkrep.sh -i acme -t ha -f p4d_edge_syd -s syd -r syd-helix-05 EXAMPLE 4 - Add a second edge server in the same site as another edge. mkrep.sh -i acme -t edge -N 2 -s syd -r syd-helix-04