# CONFIGURATION VARIABLES: # # Change the first four variables below to match the volume names on the machine # you are configuring the SDP on. # # In the SDP variable below, the sdp directory needs to contain the contents of: # //guest/perforce_software/sdp/main/... # # Do not prefix these paths with a / # Make sure these volumes exist in the filesystem. # DB1 = Root metadata during install. Paths will switch during recreate_db_checkpoint.sh run. # DB2 = Offline metadata during install # If you do not have separate volumes for root and offline, just set DB1 and DB2 to the same volume. DB1=hxmetadata1 DB2=hxmetadata2 DD=hxdepots LG=hxlogs # CN is the volume for /p4/common. It is separated here so that you can set it to a different # volume if you are installing other instances on different depots volumes than where the # common folder is located. CN=$DD # Set the remaining variables appropriately. The meaning of each should be obvious. # The SDP is typically on the same volume as the common folder, so we default to that volume. SDP=/$CN/sdp # If you are sharing the depotdata volume with a replica, change this value to TRUE SHAREDDATA=FALSE OSUSER=perforce OSGROUP=perforce # CASE_SENSITIVE settings: # 0 -- Server will run '-C1' mode, forcing case-insensitive mode on normally case-sensitive platforms # 1 -- Server will run with case sensitivity default of the underlying platform (Unix is case sensitive). CASE_SENSITIVE=1 # Admin user's account name. ADMINUSER=perforce # Admin user's password P4ADMINPASS=adminpass # Email address from which emails are sent. This must be exactly one email address. Depending on the # configuration of the email server, it may or may not be a distribution list. More strict email # server configurations require it to be an email associated with an individual account. MAILFROM="admin@example.com" # Email to receive emails from SDP scripts. This can be a comma-separated list and/or contain a distribution # list email (e.g. PerforceAdmin@MyCompany.com). If you want something other than $MAILFROM, just # remove $MAILFROM and create your own list in quotes. MAILTO=$MAILFROM # Mail Host Address MAILHOST=mail.example.com # SSL_PREFIX should be blank to not use SSL, otherwise ssl: # # If you are planning to use SSL, you need to put certificates in /p4/ssl after the SDP install or # you can generate a self signed certificate as follows: # Edit /p4/ssl/config.txt to put in the info. for your company. # Then run: /p4/common/bin/p4master_run <instance> /p4/<instance>/p4d_<instance> -Gc # For example using instance 1: # /p4/common/bin/p4master_run 1 /p4/1/bin/p4d_1 -Gc # SSL_PREFIX=ssl: # Set MASTERINSTANCE to the first instance in your installation. # This is used for linking the license and ssl dir to the other instances to avoid duplication. MASTERINSTANCE=1 P4_PORT=${SDP_INSTANCE}666 P4BROKER_PORT=${SDP_INSTANCE}667 P4WEB_PORT=808${SDP_INSTANCE} P4FTP_PORT=202${SDP_INSTANCE} # If your SDP_INSTANCE is non numeric, then uncomment the items below to set the PORT setting. # Port for this Perforce server to listen on. # P4_PORT=1666 # If your proxy listening port is different from its master listening port, set the master's target port below P4P_TARGET_PORT=1666 # Ignore the broker port if you are not running a broker. # P4BROKER_PORT=1667 # P4WEB_PORT=80 # P4FTP_PORT=21 # DNS Name or IP address of master or edge server # A DNS Name is much easier to manage for failover purposes. P4DNSNAME=DNS_name_of_master_server # Replication service user's password P4SERVICEPASS=servicepass # The server.id of the master server. Or, if configuring # an edge server or replica of an edge server, set this # to the server.id of the edge server. MASTER_ID=master.${SDP_INSTANCE} # Set SERVER_TYPE to one of: master, replica, standby, edge, broker, proxy # p4d_master - A master/commit server. # p4d_replica - A replica (readonly or forwarding, possibly filtered). # p4d_standby - A standby replica using journalcopy # p4d_edge - An edge server # p4broker - An SDP host running only a broker, with no p4d. # p4proxy - An SDP host running a proxy (maybe with a broker in front), with no p4d. SERVER_TYPE=p4d_master # Replica server.id name REPLICA_ID=replica # Email address for p4review complaints for each instance # look something like P4Review_1666@example.com. Set # the COMPLAINFROM_PREFIX (e.g. "P4Review") and # COMPLAINFROM_DOMAIN (e.g. "example.com)" here. Instance # specific values are substituted below. COMPLAINFROM_DOMAIN=example.com COMPLAINFROM="${SSL_PREFIX}${P4DNSNAME}:${P4_PORT}_P4Review\@${COMPLAINFROM_DOMAIN}"
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#21 | 30333 | C. Thomas Tyler |
Added HxCheckpoints, optional setting to store checkpoints on a separate volume from depots. Default is same value as HxDepots. This changes makes the SPD more adaptable to customers, especially using NFS, that prefer to have separate mounts for checkpoints vs. those for archive files. The original SDP design was to have everything that needed to be backed up (i.e. metadata checkpoints and the versioned file tree) on a single volume. This is still the preferred mode, but allowing checkpoints and archives to be separate is useful for some. #review-30334 |
||
#20 | 29851 | C. Thomas Tyler |
Changed default value of P4MASTERHOST from an illustrative sample value, DNS_name_of_master_server_for_this_instance, to a value that will work on Day 1: localhost. Also removed outdated text about the P4MASTERHOST value; it should ALWAYS refer to the commit server. #review-29852 |
||
#19 | 29849 | C. Thomas Tyler |
Updates to configure_new_server.sh: * Added logic to start service so this script can now be run if the service is down at the start. * The 'security' configurable is now set to 4 (was unset). * Enhanced error handling. * Added logic to more fully initialize a new/empty server: - Create perforce P4USER (if there is only a single user on the system). - Create Automation group. - Initialize basic Protections. Updates to configure_new_server.bat: * Added logic to start service so this script can now be run if the service is down at the start. * The 'security' configurable is now set to 4 (was unset). In mkdirs.cfg, change the default password to one good enough to be used with `security=4`. #review-29850 |
||
#18 | 28687 | Robert Cowham | Clarify comments for P4MASTERHOST | ||
#17 | 28192 | C. Thomas Tyler |
Added new optional PERMS setting in mkdirs.cfg, to allow opening of SDP perms when mkdirs.sh is used. This is useful for customer environments (mainly test environments). Documentation for the feature is included in the mkdirs.cfg file. Default behavior is unchanged. |
||
#16 | 27977 | Robert Cowham | Reduce unnecessary repetition in domain names for many instances. | ||
#15 | 27722 | C. Thomas Tyler |
Refinements to @27712: * Resolved one out-of-date file (verify_sdp.sh). * Added missing adoc file for which HTML file had a change (WorkflowEnforcementTriggers.adoc). * Updated revdate/revnumber in *.adoc files. * Additional content updates in Server/Unix/p4/common/etc/cron.d/ReadMe.md. * Bumped version numbers on scripts with Version= def'n. * Generated HTML, PDF, and doc/gen files: - Most HTML and all PDF are generated using Makefiles that call an AsciiDoc utility. - HTML for Perl scripts is generated with pod2html. - doc/gen/*.man.txt files are generated with .../tools/gen_script_man_pages.sh. #review-27712 |
||
#14 | 26986 | C. Thomas Tyler |
Refinements to mkdirs.sh and mkdirs.cfg, per code review: https://swarm.workshop.perforce.com/reviews/26962 In mkdirs.cfg: * Added comments about proper use of DB1/DB2. * Added comment about coming removal of P4FTP/P4WEB settings. In mkdirs.sh v4.1.1: * Replaced -M with set of -M* flags to specify mount points. * Completed implementation of -M* flags. * Enhanced examples in usage function. * Added clarification of difference between '-n' and '-p'. * Fixed some typos. #review @rcowham |
||
#13 | 26982 | C. Thomas Tyler |
mkdirs.sh v4.1.0: * Accounted for directory structure change of Maintenance to Unsupported. * Added standard command line processing with '-h' and '-man' doc flags, and other flags (all documented). * Added in-code docs and updated AsciiDoc. * Enhanced '-test' mode to simulate /hx* mounts. * Enhanced preflight testing, and fixed '-test' mode installs. * Added support for installing to an alternate root directory. * Added '-s <ServerID>' option to override REPLICA_ID. * Added '-S <TargetServerID>' used for replicas of edge servers. * Added '-t <server_type>' option to override SERVER_TYPE. * Added '-M' option to override mount points. * Added '-f' fast option to skip big chown/chmod commands, and moved those commands near the end as well. verify_sdp.sh v5.9.0: * Added check for /p4/Version file, and checked that other legacy SDP methods of checking version * Added sanity check for crontab. * Added 'test skip' mechanism to skip certain tests: - crontab: Skip crontab check. Use this if you do not expect crontab to be configured, perhaps if a different scheduler is used. - license: Skip license related checks. - version: Skip version checks. - excess: Skip checks for excess copies of p4d/p4p/p4broker in PATH. * Added VERIFY_SDP_SKIP_TEST_LIST setting ton instance_vars.template, to define a standard way to have verify_sdp.sh always skip certain tests for a site. * Extended '-online' checks to check for bogus P4MASTERPORT, a common config error. Update test_SDP.py: * Adjusted test suite to account for various changes in mkdirs.sh. * Added 'dir' parameter to run_cmd() and sudo_cmd(), to run a command from a specified directory (as required to test new mkdirs.sh) * Added check_links() similar to existing check_dirs() function. === Upgrade Process Changes === Made /p4/common/bin/p4d/p4/p4broker/p4p shell script rather than binary. This changes the way SDP new binaries are staged for upgrade. For safety, exes are now staged to a director outside the PATH, the /p4/sdp/exes folder. A new 'get_latest_exes.sh' script simplifies the task of pulling executables from the Perforce FTP server. This can be used 'as is' for environments with outbound internet access, and is useful in any case to describe now to acquire binaries. This addresses an issue where a p4d binary staged for a future upgrade might be called before the actual upgrade is performed. upgrade.sh v4.0.0: * All preflight checks are now done first. Added '-p' to abort after preflight. * Added '-n' to show what would be done before anything is executed. * Minimalist logic to start/stop only servers that are upgrade, and apply upgrades only as needed. * Staging of exes for upgrade is now separate from /p4/common/bin * Improved in-code docs, added '-h' and '-man' options. * Retained pre/post P4D 2019.1 upgrade logic. |
||
#12 | 26732 | Robert Cowham |
Change default for DB1/DB2 to /hxmetadata with comment that these can be seperated Add a couple of other clarifying comments. |
||
#11 | 26718 | Robert Cowham |
Rename P4MASTER to P4MASTERHOST for clarity with comments in: - mkdirs.cfg/mkdirs.sh - p4_<instance>.vars - other files which reference Remove unnecessary sed for p4p.template |
||
#10 | 26529 | C. Thomas Tyler |
Corrected comments in mkdirs.cfg for P4MASTER_ID. The P4MASTER_ID should only be set to the master server (aka commit-server in a topology including edge servers). In cases where the P4TARGET of a replica-of-an-edge is needed, the P4TARGET value can be calculated dynamically. We leave P4MASTER_ID to be statically configured, as typically it is only configured once in the life of an instance. Other aspects of the topology are dynamic and require dynamic detection. This is a non-functional, comment-only change. #review-26530 @ashaikh |
||
#9 | 25363 | C. Thomas Tyler |
Fixed bug where /p4/N/bin ownership is not set correctly (owned by root). Fixed issue with missing symlink creation for replicas. Made compliant with shellcheck.sh v0.6.0, driving many changes and fixing potential glob-expansion issues. Fixed an issue with '-test' not setting DB1/DB2. Removed unused/obsolete MD def'n. Added 'p4d_ha' server type to distinguish from existing 'p4d_standby' type, and added comments to mkdirs.cfg to note that p4d_standby is for pre-2018.1 servers, and p4d_ha is for 2018.1+. The distinction is that P4D 2018.1 introduced rpl.journalcopy.location, which changes P4D behaviors and corresponding best practice for placement of the actively pulled journal file for 'journalcopy' replicas. For pre-2018.1, we use /hxlogs/p4/N/journals.rep, and for 2018.1+ with rpl.journalcopy.location=1 set, we us /hxdepots/p4/N/checkpoints.<tag>. Normalized inconsistent indentation. |
||
#8 | 25190 | Robert Cowham |
Change to make instance dir a directory under /p4 (so on root volume). This avoids potential problems where /hxdepots is shared on multiple instances. It also improves performance often. Left the option to put /p4/common on a shared volume if required. |
||
#7 | 25142 | Robert Cowham | Clarify comments - no other change | ||
#6 | 23266 | C. Thomas Tyler |
Fixes and Enhancements: * Enabled daily_checkpoint.sh operate on edge servers, to keep /p4/N/offline_db current on those hosts for site-local recovery w/o requiring a site-local replica (though having a site-local replica can still be useful). * Disabled live_checkpoint.sh for edge servers. * More fully support topologies using edge severs, in both geographically distributed and horizaontal scaling "wokspace server" solutions. * Fix broken EDGESERVER value definition. * Modified name of SDP counter that gets set when a checkpoint is taken to incorporate ServerID, so now the counter name will look like lastSDPCheckpoint.master.1, or lastSDPCheckpoint.p4d_edge_sfo, rather than just lastSDPCheckpoint. There will be multiple such counters in a topology that uses edge servers, and/or which takes checkpoints on replicas. * Added comments for all functions. For the master server, journalPrefix remains: /p4/N/checkpoints/p4_N The /p4/N/checkpoints is reserved for writing by the master/commit server only. For non-standby (possibly filtered) replicas and edge serves, journalPrefix is: /p4/N/checkpoints.<ShortServerID>/p4_N.<ShortServerID> Here, ShortServerID is just the ServerID with the 'p4d_' prefix trimmed, since it is redundant in this context. See mkrep.sh, which enshines a ServerID (server spec) naming standard, with values like 'p4d_fr_bos' (forwarding replica in Boston) and p4d_edge_blr (Edge server in Bangalore). So the journalPrefix for the p4d_edge_bos replica would be: /p4/N/checkpoints.edge_bos/p4_N.edge_bos For "standby" (aka journalcopy) replicas, journalPrefix is set to /p4/N/journals.rep. which is written to the $LOGS volume, due to the nature of standby replicas using journalPrefix to write active server logs to pre-rotated journals. Some take-away to be updated in docs: * The /p4/N/checkpoints folder must be reserved for checkpoints that originate on the master. It should be safe to rsync this folder (with --delete if desired) to any replica or edge server. This is consistent with the current SDP. * I want to change 'journals.rep' to 'checkpoints.<ShortServerID>' for non-standby replicas, to ensure that checkpoints and journals taken on those hosts are written to a volume where they are backed up. * In sites with multiple edge serves, some sharing achive files ('workspace servers'), multiple edge servers will share the same SAN. So we one checkpoints dir per ServerID, and we want that dir to be on the /hxdepots volume. Note that the journalPrefix for replicas was a fixed /p4/N/journals.rep. This was on the /hxlogs volume - a presumably fast-for-writes volume, but typically NOT backed up and not very large. This change puts it under /p4/N/checkpoints.* for edge servers and non-standby replicas, but ensures other replica types and edge servers can generate checkpoints to a location that is backed up and has plenty of storage capacity. For standby replicas only (which cannot be filtered), the journalPrefix remains /p4/N/journals.rep on the /hxlogs volume. |
||
#5 | 22957 | C. Thomas Tyler |
Merged two changes initiated on main down to: Change @22826 by robert_cowham: Ignore some files on Mac. Change @22950 by awkan/ttyler: Provide an override for difference between proxy listening port and target port |
||
#4 | 22571 | C. Thomas Tyler |
Changed default (and defacto standard) value for server.id of a master server to make it unique by incorporating the SDP_INSTANCE name. So for instance 'fgs', the server.id would not be 'master.fgs' instead of just 'master'. This is necessary to enable remote depots, which won't work if two servers involved in remote depot sharing have the same server.id. Affects Unix SDP. |
||
#3 | 22333 | Russell C. Jackson (Rusty) |
The service user always needs to be defined since the proxy base script uses it as well. Added note about making sure to leave replica FALSE when setting up a proxy. If you don't the instance vars script will create an empty database by running the cshow commands. |
||
#2 | 22145 | Robert Cowham | Update mkdirs to help testing. | ||
#1 | 22070 | Russell C. Jackson (Rusty) |
Pull the configuration items out into mkdirs.cfg and source that file in mkdirs.sh. Makes it a little cleaner, and less likely for someone to mess up the mkdirs.sh script. Also preps for potential future configure_sdp.sh script to ask questions and create the mkdirs.cfg file. |