USAGE for mkrep.sh v3.2.1:
mkrep.sh -t <Type> -s <Site_Tag> -r <Replica_Host> [-f <From_ServerID>] [-os] [-p] [-N <N>] [-i <SDP_Instance>] [-L <log>] [-v<n>] [-n] [-D]
or
mkrep.sh [-h|-man|-V]
DESCRIPTION:
This script simplifies the task of creating Helix Core replicas and
edge servers, and helps ensure they are setup with best practices.
This script executes as two phases. In Phase 1, this script does all
the metadata configuration to be executed on the master server that
must be baked into a seed checkpoint for creating the replica/edge.
This essentially captures the planning for a new replica, and can be
done before the physical infrastructure (e.g. hardware, storage, and
networking) is ready. Phase 1, fully automated by this script, takes
only seconds to run.
In Phase 2, this script provides information for the manual steps
needed to create, transfer, and load seed checkpoints onto the
replica/edge. The guidance is specific to type of replica created,
based on the command line flags provided to this script. This
processing can take a while for large data sets, as it involves
creating and transporting checkpoints.
Before using this script, a set of geographic site tags must be defined.
See the FILES: below for details on a site tags.
This script adheres to the these SDP Standards:
* Server Spec Naming Standard: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Guide.Unix.html#_server_spec_naming_standard
* Journal Prefix Standard: https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/doc/SDP_Guide.Unix.html#_the_journalprefix_standard
In Phase 1, this script does the following to help create a replica or
edge server:
* Generates the server spec for the the replica.
* Generates a server spec for master server (if needed).
* Sets configurables ('p4 configure' settings) for replication.
* Selects the correct 'Services' based on replica type.
* Creates service user for the replica, and sets a password.
* Creates service user for the master (if needed), and sets a password.
* Adds newly created service users to the group 'ServiceUsers'.
* Verifies the group ServiceUsers is granted super access in the
protections table (and with the '-p', updates Protections).
After these steps are completed, in Phase 2, detailed instructions are
presented to guide the user through the remaining steps needed to complete
the deployment of the replica. This starts with creating a new
checkpoint to capture all the metadata changes made by this
script in Phase 1.
SERVICE USERS:
Service users created by this type are always of type 'service',
and so will not consume a licensed seat.
Service users also have an 'AuthMethod' of 'perforce' (not
'ldap') as is required by 'p4d' for 'service' users. Passwords
set for service users are long 32 character random strings
that are not stored, as they are never needed. Login tickets for
service users are generated using: p4login -service -v
OPTIONS:
-t <Type>[N]
Specify the replica type tag. The type corresponds to the 'Type:' and
'Services:' field of the server spec, which describes the type of services
offered by a given replica.
Valid type values are:
* ha: High Availability standby replica, for 'p4 failover' (P4D 2018.2+)
* ham: High Availability metadata-only standby replica, for 'p4 failover' (P4D 2018.2+)
* ro: Read-Only standby replica. (Discouraged; Use 'ha' instead for 'p4 failover' support.)
* rom: Read-Only standby replica, Metadata only. (Discouraged; Use 'ham' instead for 'p4 failover' support.)
* fr: Forwarding Replica (Unfiltered).
* fs: Forwarding Standby (Unfiltered).
* frm: Forwarding Replica (Unfiltered, Metadata only).
* fsm: Forwarding Standby (Unfiltered, Metadata only).
* ffr: Filtered Forwarding Replica. Not a valid failover target.
* edge: Edge Server. Filtered by definition.
Replicas with 'standby' are always unfiltered, and use the 'journalcopy'
method of replication, which copies a byte-for-byte verbatim journal file
rather than one that is merely logically equivalent.
The tag has several purposes:
1. Short Hand. Each tag represents a combination of 'Type:' and fully
qualified 'Services:' values used in server specs.
2. Distillation. Only the most useful Type/Services combinations have a
shorthand form
3. For forwarding replicas, the name includes the critical distinction of
whether any replication filtering is used; as filtering of any kind disqualifies
a replica from being a potential failover target. (No such distinction is
needed for edge servers, which are filtered by definition).
-s <Site_Tag>
Specify a geographic site tag indicating the location and/or data center where
the replica will physically be located. Valid site tags are defined in the site
tags file:
/p4/common/config/SiteTags.cfg
A sample SiteTags.cfg file that is here:
/p4/common/config/SiteTags.cfg.sample
-r <Replica_Host>
Specify the DNS name of the server machine on which the new replica will
run. This is used in the 'ExternalAddress:' field of the replica's
ServerID, and also used in instructions to the user for steps after
metadata configuration is done by this script.
-f <From_ServerID>
Specify ServerID of the P4TARGET server from which we are replicating.
This is used to populate the 'ReplicatingFrom' field of the server
spec. The value must be a valid ServerID.
This option should be used if the target is something other than the
master. For example, to create an HA replica of an edge server, you might
specify something like '-f p4d_edge_syd'.
-os Specify the '-os' option to overwrite an exising server spec. By
default, this script will abort of the server spec to be generated
already exists on the Helix Core server. Specify this option to
overwrite the existing server spec.
-p This script performs a check to ensure that the Protections table grants
super access to the group ServiceUsers.
By default, an error is displayed if the check fails, i.e. if super user
access for the group ServiceUsers cannot be verified. This is
because, by default, we want to avoid making changes to the Protections
table. Some sites have local policies or custom automation that requires
site-specific procedures to update the Protections table.
If '-p' is specified, an attempt is made to append the Protections table
an entry like:
super group ServiceUsers * //...
-N <N>
Specify '-N <N>', where N is an integer. This is used to indicate that
multiple replicas of the same type are to be created at the same site.
The value specified with '-N' must be a numeric value. Left-padding with
zeroes is allowed. For example, '-N 04' is allowed, and 'N A7' is not
(as it is not numeric).
This affects the ServerID to ee generated. For example, the options
'-t edge -s syd' would result in a ServerID of p4d_edge_syd. To
create a second edge in the same site, use '-t edge -s syd -N 2' to
generate p4d_edge2_syd.
-i <SDP_Instance>
Specify the SDP Instance. If not specifed and the SDP_INSTANCE environment
is defined, that value is used. If SDP_INSTANCE is not defined, the
'-i <SDP_Instance>' argument is required.
-v<n> Set verbosity 1-5 (-v1 = quiet, -v5 = highest).
-L <log>
Specify the path to a log file, or the special value 'off' to disable
logging. By default, all output (stdout and stderr) goes in the logs
directory referenced by $LOGS environment variable, in a file named
mkrep.<timestamp>.log
NOTE: This script is self-logging. That is, output displayed on the screen
is simultaneously captured in the log file. Do not run this script with
redirection operators like '> log' or '2>&1', and do not use 'tee.'
-n No-Op. Prints commands instead of running them.
-D Set extreme debugging verbosity.
HELP OPTIONS:
-h Display short help message
-man Display man-style help message
-V Display version info for this script and its libraries.
FILES:
This Site Tags file defines the list of valid geographic site tags:
/p4/common/config/SiteTags.cfg
The contains one-line entries of the form:
<tag>: <description>
where <tag> is a short alphanumeric tag name for a geographic location,
data center, or other useful distinction. This tag is incorporated into
the ServerID of replicas or edge servers created by this script. Tag
names should be kept short, ideally no more than about 5 characters in
length.
The <description> is a one-line text description of what the tag
refers to, which may contain spaces and ASCII punctuation.
Blank lines and lines starting with a '#' are considered comments
and are ignored.
REPLICA SERVER MACHINE SETUP:
The replica/edge server machine must be have the SDP structure installed,
either using the mkdirs.sh script included in the SDP, or the Helix
Installer for 'green field' installations.
When setting up an edge server, a replica of an edge server, or filtered
replica, confirm that the JournaPrefix Standard (see URL above) structure
has the separate checkpoints folder as identified in the 'Second Form' in
the standard. A baseline SDP structure can typically be extended by running
commands like like these samples (assuming a ServerID of p4d_edge_syd or
p4d_ha_edge_syd):
mkdir /hxdepots/p4/1/checkpoints.edge_syd
cd /p4/1
ln -s /hxdepots/p4/1/checkpoints.edge_syd
CUSTOM PRE- AND POST- OPERATION AUTOMATION HOOKS:
This script can execute custom pre- and post- processing scripts. This
can be useful to incorporate site-specifc elements of replica setup.
If the file /p4/common/site/mkrep/pre-mkrep.sh exists and is
executable, it will be executed before mkrep.sh processing. If the file
/p4/common/site/mkrep/post-mkrep.sh exists and is executable,
it will be executed after mkrep.sh processing.
Pre- and post- processing scripts are called with the same command line
arguments passed to this mkrep.sh script.
The pre- and post- processing scripts can use or ignore arguments as
needed, though it is required to implement the '-n' flag to operate in
preview mode, taking no actions that affect data (just as this script
behaves).
Pre- and post- processing scripts are expected to exit with a zero exit
code to indicate success, and non-zero to indicate failure.
The custom pre-processing script is executed after standard preflight
checks complete successfully. If a custom pre-processing script
indicates a failure, processing is aborted before standard mkrep.sh
processing occurs.
The post-processing custom script is executed after the standard
mkrep.sh processing is successful. If a post-processing custom script
is detected, the instructions that would be provided to the user in
Phase 2 are not displayed, as it is expected that the custom post-
processing will alter or handle these steps.
Success or failure of pre- and post- processing scripts is reported in
the log. These scripts do not require independent logging, as all
standard and error output is captured in the log of this mkrep.sh
script.
TIP: Be sure to fully test custom scripts in a test environment
before incorporating them into production systems.
EXAMPLES:
EXAMPLE 1 - Set up a High Availability (HA) Replica of the master.
Add an HA replica to instance 1 to run on host bos-helix-02:
mkrep.sh -i 1 -t ha -s bos -r bos-helix-02
EXAMPLE 2 - Add an Edge Server to the topology.
Add an Edge server to instance acme to run on host syd-helix-04:
mkrep.sh -i acme -t edge -s syd -r syd-helix-04
EXAMPLE 3 - Setup an HA replica of an edge server.
Add a HA replica of the edge server to instance acme to run on host syd-helix-05:
mkrep.sh -i acme -t ha -f p4d_edge_syd -s syd -r syd-helix-05
EXAMPLE 4 - Add a second edge server in the same site as another edge.
mkrep.sh -i acme -t edge -N 2 -s syd -r syd-helix-04