SDP Release Notes

Preface

Please Give Us Feedback

Perforce welcomes feedback from our users. Please send any suggestions for improving this document or the SDP to consulting@perforce.com.

1. Overview

These release notes document changes to the SDP.

There are two flavors of the SDP, Unix and Windows.

For UNIX/Linux, see: SDP Guide for UNIX/Linux

For Windows, see: SDP Guide for Windows

For information on upgradeing the SDP to the 2020.1 from prior releases on Linux, see the SDP Legacy Upgrade Guide (for Unix).

2. What’s New in SDP 2020.1.

In a nutshell, here is what the SDP r2020.1 release is all about:

Improved Documentation. The SDP documentation has been reformatted to AsciiDoc. Documentation coverage has been expanded, new content added, and documentation for various scripts is now included in the main documentation.
Publishing of the new JournalPrefix Standard.
Updating of the Server Spec Naming Standard.
Bug fixes and improvements to load_checkpoint.sh.
Bug fixes and improvements to p4verify.sh. The -recent flag is fixed and verification of shelves has improved.
Bug fixes and improvements to mkrep.sh, which adheres to documented standards.
Improved Helix Core Upgrades. The upgrade.sh script that upgrades p4, p4d, p4broker, and p4p has been greatly improved, and a new get_helix_binaries.sh script added. Documentation now covers how to do Global Topology "Outer to Inner" upgrades.
Improved Self-Verification. The verify_sdp.sh script has been improved to do many more checks. This script is also relied on by other SDP scripts to ensure things are in a good start before they start their processing. The refresh_P4ROOT_from_offline_db.sh and upgrade.sh scripts use verify_sdp.sh.
The systemd interactions have been improved in a few key ways. Notably, on systems in which systemd is configured, a safety check now ensure that systemd is used, so that systemd’s concept of service status is reliable. The format of the systemd unit files changes with this release.

More details is in the Chapter 5, Change History below.

3. What’s Coming Next in SDP 2021.1?

A key feature of the coming SDP 2021.1 release will be addition of automation to greatly simplify upgrades of the SDP itself. This will apply to SDP versions starting from this 2020.1 release going to the 2021.1 and future releases. Starting in the 2021.1 release, SDP ugprades will be similar to p4d upgrades in that whatever the current version is will always be able to upgrade from prior releases (going back to this 2020.1 release).

4. SDP Health Checks

If you need to contact Perforce Support to analyze an SDP issue on UNIX/Linux, you can use the sdp_health_check.sh script. This script is not included in the SDP, but it can be used with all versions of the SDP dating back to 2007.

To gather health check information, do the following:

cd /p4/1/tmp
curl -s -k -O https://swarm.workshop.perforce.com/projects/perforce-software-sdp/download/tools/sdp_health_check.sh
chmod +x sdp_health_check.sh
./sdp_health_check.sh

This script will produce a log file that can be provided to Perforce Support to help diagnose configuration issues and other problems.

5. Change History

Released: Rev. SDP/MultiArch/2020.1/27114 (2020/12/31).

Jobs Fixed since change 26494 up to 27114 (F=Feature, B=Bug):

SDP-246 (B): Sourcing p4_vars hangs if live_checkpoint.sh is running.
SDP-373 (B): Typo on page 22 of guide (Section 4.2): modifed vs modified
SDP-428 (F): Add safety check to ensure systemd is used if required.
SDP-432 (F): Enhance distinction of what is Supported and Unsupported.
SDP-439 (B): DOC: SDP Guide for setting up a replica is missing key items.
SDP-460 (B): Script 'broker_rotate.sh' is not mentioned in the SDP Guide (Unix).
SDP-461 (B): Script 'edge_dump.sh' is not mentioned in the SDP Guide (Unix).
SDP-462 (B): Script 'edge_shelf_replicate.sh' is not mentioned in the SDP Guide (Unix).
SDP-463 (B): Script 'edge_vars' is not mentioned in the SDP Guide (Unix).
SDP-464 (B): Script 'gen_default_broker_cfg.sh' is not mentioned in the SDP Guide (Unix).
SDP-465 (B): Script 'htd_move_logs.sh' is not mentioned in the SDP Guide (Unix).
SDP-466 (B): Script 'journal_watch.sh' is not mentioned in the SDP Guide (Unix).
SDP-467 (B): Script 'kill_idle.sh' is not mentioned in the SDP Guide (Unix).
SDP-468 (B): Script 'load_checkpoint.sh' is not mentioned in the SDP Guide (Unix).
SDP-469 (B): Script 'mkrep.sh' is not mentioned in the SDP Guide (Unix).
SDP-470 (B): Script 'p4broker_base' is not mentioned in the SDP Guide (Unix).
SDP-471 (B): Script 'p4dstate.sh' is not mentioned in the SDP Guide (Unix).
SDP-472 (B): Script 'p4ftpd_base' is not mentioned in the SDP Guide (Unix).
SDP-473 (B): Script 'p4login-super.sh' is not mentioned in the SDP Guide (Unix).
SDP-474 (B): Script 'p4p_base' is not mentioned in the SDP Guide (Unix).
SDP-475 (B): Script 'p4pcm.pl' is not mentioned in the SDP Guide (Unix).
SDP-476 (B): Script 'p4review2.py' is not mentioned in the SDP Guide (Unix).
SDP-477 (B): Script 'p4sanity_check.sh' is not mentioned in the SDP Guide (Unix).
SDP-478 (B): Script 'p4web_base' is not mentioned in the SDP Guide (Unix).
SDP-481 (B): Script 'ps_functions.sh' is not mentioned in the SDP Guide (Unix).
SDP-482 (B): Script 'purge_revisions.sh' is not mentioned in the SDP Guide (Unix).
SDP-483 (B): Script 'recover_edge.sh' is not mentioned in the SDP Guide (Unix).
SDP-484 (B): Script 'replica_cleanup.sh' is not mentioned in the SDP Guide (Unix).
SDP-485 (B): Script 'replica_status.sh' is not mentioned in the SDP Guide (Unix).
SDP-486 (B): Script 'request_replica_checkpoint.sh' is not mentioned in the SDP Guide (Unix).
SDP-487 (B): Script 'rotate_journal.sh' is not mentioned in the SDP Guide (Unix).
SDP-488 (B): Script 'template.pl' is not mentioned in the SDP Guide (Unix).
SDP-489 (B): Script 'template.py' is not mentioned in the SDP Guide (Unix).
SDP-490 (B): Script 'template.py.cfg' is not mentioned in the SDP Guide (Unix).
SDP-491 (B): Script 'template.sh' is not mentioned in the SDP Guide (Unix).
SDP-492 (B): Script 'update_limits.py' is not mentioned in the SDP Guide (Unix).
SDP-493 (B): Script 'verify_sdp.sh' is not mentioned in the SDP Guide (Unix).
SDP-495 (B): SDP Guide mentions now-deleted/obsolete script.
SDP-496 (B): Script 'backup_functions.sh' is mentioned in the SDP Guide (Unix).
SDP-497 (B): Script 'p4d_base' is mentioned in the SDP Guide (Unix).
SDP-498 (B): Script 'p4verify.sh' is mentioned in the SDP Guide (Unix).
SDP-499 (B): Script 'run_if_edge.sh' is mentioned in the SDP Guide (Unix).
SDP-500 (B): Script 'run_if_master.sh' is mentioned in the SDP Guide (Unix).
SDP-501 (B): Script 'run_if_replica.sh' is mentioned in the SDP Guide (Unix).
SDP-502 (B): Script 'sync_replica.sh' is mentioned in the SDP Guide (Unix).
SDP-503 (B): SDP Guide mentions some scripts by most of the filename, not all of it.
SDP-504 (B): SDP Guide mentions "weekly_checkpoint" script under "daily_checkpoint":
SDP-507 (B): p4d_1.service includes wrong doc link
SDP-508 (B): p4broker_1.service includes wrong doc link
SDP-509 (B): "sdp-administrator-guide.docx" needs to be removed or updated.
SDP-512 (B): upgrade.sh needs clearer documented explanation
SDP-523 (B): Missing 'failover' document
SDP-535 (B): False-positive on journal corruption check w/multi-line journal entries.
SDP-538 (F): Document systemd interactions with SDP init scripts.
SDP-541 (F): Make 'p4d' rather than 'p4d_N' safe to run.
SDP-543 (B): The configure_new_server.sh failed with unbound LOGFILE variable error.
SDP-545 (B): DOC: PDF contains "Reference source not found" errors.
SDP-546 (B): Documentation out of date.
SDP-548 (B): Standby won’t start pull if journal.NNN exists.
SDP-553 (B): Remove 'Operating System' from configurable rejectList.
SDP-558 (B): Fix bug with '-recent' flag in p4verify.sh.
SDP-559 (B): Fix load_checkpoint.sh init issue if '-s <ServerID>' is used.
SDP-564 (F): Normalized SDP Versioning will start in the SDP r20.1 release.
SDP-565 (F): Enhance verify_sdp.sh to allow optional skip of some tests.
SDP-566 (F): Add Support-recommended sysctl settings to SDP.
SDP-567 (F): Add setting for server.extensions.dir.
SDP-572 (B): Fixed issue where No Checkpoints reported if 'find' doesn’t support '-printf'.
SDP-573 (B): mkdirs.sh is not respecting MASTER_ID from config file
SDP-574 (B): daily_checkpoint.sh hangs if journalPrefix is wrong.
SDP-575 (B): The stop_p4d() function reports 'missing server.pid' error with systemd.
SDP-576 (F): Added documentation for handling multiple broker configs with systemd.
SDP-579 (B): Edge recover script fails to run if p4_vars not sourced first.
SDP-581 (F): Added optional check that P4MASTER_ID starts with 'master' to verify_sdp.sh.
SDP-583 (E): Enhance verify_sdp.sh and upgrade.sh to handle proxy-only and broker-only machines.

Released: Rev. SDP/MultiArch/2019.3/26494 (2020/04/23).

Jobs Fixed since change 26478 up to 26494 (F=Feature, B=Bug):

SDP-529 (F): Add ability to limit specific commands to one-per-user, e.g. populate.
SDP-531 (B): The recover_edge.sh script needed updating to use new start/stop functions.

Released: Rev. SDP/MultiArch/2019.3/26478 (2020/04/12).

Jobs Fixed since change 26472 up to 26478 (F=Feature, B=Bug):

SDP-527 (B): Patch to rotate_journal.sh and add test coverage for it.
SDP-528 (F): Enhance dynamic check to work with replicas that do not set journalPrefix.

Released: Rev. SDP/MultiArch/2019.3/26472 (2020/04/10).

Jobs Fixed since change 26468 up to 26472 (F=Feature, B=Bug):

SDP-526 (F): Use dynamic journalPrefix detection for non-master servers.

Released: Rev. SDP/MultiArch/2019.3/26468 (2020/04/10).

Jobs Fixed since change 26407 up to 26468 (F=Feature, B=Bug):

SDP-446 (B): wrong variables used in recreate_offline_db_files
SDP-519 (B): Fixed missing variable initialization in load_checkpoint.sh.
SDP-520 (F): Add login and trust logic to load_checkpoint.sh.
SDP-521 (F): Have load_checkpoint.sh handle P4SSLDIR and cert creation automatically.
SDP-522 (F): Have load_checkpoint.sh check for a license file.
SDP-524 (B): Fix refresh_P4ROOT_from_offline_db.sh to work on a non-edge replica.
SDP-525 (B): Remove parallel checkpoint scripts temporarily.

Released: Rev. SDP/MultiArch/2019.3/26407 (2020/03/28).

Jobs Fixed since change 26400 up to 26407 (F=Feature, B=Bug):

SDP-427 (F): Use systemd consistently if availabile for p4d start/stop.

Released: Rev. SDP/MultiArch/2019.3/26400 (2020/03/28).

Jobs Fixed since change 26239 up to 26400 (F=Feature, B=Bug):

SDP-427 (F): Use systemd consistently if availabile for p4d start/stop.
SDP-431 (F): Make p4d startup resistant to timing issues for reboots & fast
SDP-436 (F): Provide snapshot hook for daily_checkpoint.sh.
SDP-444 (F): Add script to safely refresh P4ROOT db.* from offline_db.
SDP-451 (F): Added '-f <P4TARGET>' flag to mkrep.sh.
SDP-452 (F): Enhance mkrep.sh to instruct using edge_dump.sh for edge servers.
SDP-453 (F): For filtered replicas, have mkrep.sh instruct using '-P ServerID'.
SDP-454 (B): verify_sdp.sh should handle being run as root
SDP-455 (F): Add license expiration check to verify_sdp.sh.
SDP-510 (B): Fix issue with 'clean start' test suite run due to Python symlink issues.

Released: Rev. SDP/MultiArch/2019.3/26239 (2020/01/08).

Jobs Fixed since change 26159 up to 26239 (F=Feature, B=Bug):

SDP-423 (B): slight typo in comment /p4/INSTANCe/checkpoints
SDP-430 (B): Fixed typo causing os_tweaks.sh not to recognize Ubuntu 18.
SDP-437 (F): Enhance mkdirs.sh for standby replicas to match other replica types.
SDP-441 (B): Upgrade.ps1 fails with error for Windows SDP
SDP-443 (B): Fix in maint scripts to pickup SDP_INSTANCE if no arg passed.

Released: Rev. SDP/MultiArch/2019.3/26159 (2019/11/06).

Jobs Fixed since change 25949 up to 26159 (F=Feature, B=Bug):

SDP-448 (F): Enable 'p4 submit -b' by setting submit.allowbgtransfer=1 for P4D 2019.1.
SDP-352 (F): Eliminate excessive journal rotations on p4d startup.
SDP-412 (B): Backup failing as logs cannot be rotated as the server is too busy.
SDP-424 (B): sync_replica.sh fails on standby replica w/rpl.journalcopy.location=1.
SDP-425 (B): mkrep.sh fails for replica of replica due to ReplicatingFrom field hard coded to master.1
SDP-426 (F): Add -f flag to p4d -jr on SDP for Windows.

Released: Rev. SDP/MultiArch/2019.2/25949 (2019/08/12).

Jobs Fixed since change 25938 up to 25949 (F=Feature, B=Bug):

SDP-415 (B): Fixed typo in scripts for handling edge servers.

Released: Rev. SDP/MultiArch/2019.2/25938 (2019/08/05).

Jobs Fixed since change 25923 up to 25938 (F=Feature, B=Bug):

SDP-414 (B): Fixed issue with safety check in edge processing scripts.

Released: Rev. SDP/MultiArch/2019.2/25923 (2019/08/05).

Jobs Fixed since change 25594 up to 25923 (F=Feature, B=Bug):

SDP-398 (F): Disallow replicas from being initially mandatory per P4D 2019.1 requirement.
SDP-400 (B): Fixed p4login bug where P4PORT value can have a value of Unset.
SDP-401 (F): Enanced replica_status.sh to check archive replication.
SDP-403 (B): Fixed root-owned symlinks in /p4/N after running mkdirs.sh.
SDP-404 (F): Update edge scripts to support 2019.1+ changes to edge servers.
SDP-405 (F): Enhanced load_checkpoint.sh to handle case-insensitive checkpoints.
SDP-410 (B): [R]CMDEXITCODE not set when [r]run() called with 'grep string' parameter.
SDP-413 (F): Add script to request checkpoint on edge/replica.

Released: Rev. SDP/MultiArch/2019.2/25594 (2019/05/02).

Jobs Fixed since change 25480 up to 25594 (F=Feature, B=Bug):

SDP-356 (F): Reduce SDP Core to contain only software supported by Perforce Support.
SDP-369 (F): Added flags to p4verify.sh to verify by type like shelved.
SDP-374 (B): Fixed off-by-three-orders-of-magnitude error in proxy cleanup script.
SDP-378 (F): Removed journalPrefix as command line paramter during journal rotation, deferring to db.config values.
SDP-380 (F): Removed journalPrefix as command line paramter during journal rotation, deferring to db.config values.
SDP-383 (F): Added '-a' flag meant to be equivalent of most common set of flags.
SDP-384 (B): Fixed failure to cleanup old logs in sdp_sync.sh on Ubuntu Linux.
SDP-392 (B): upgrade.sh must handle new 19.1 features
SDP-393 (B): Fixed perms on purge_revisions.sh and journal_watch.sh.
SDP-394 (F): Added support for wildcards/globbing to purge_revisions.sh scripts.
SDP-395 (F): Added support for 2019.1 upgrades and testing of same.

Released: Rev. SDP/MultiArch/2019.1/25480 (2019/04/11).

Jobs Fixed since change 25386 up to 25480 (F=Feature, B=Bug):

SDP-382 (B): Fixed missing 'exit 1' in p4d_base after 'p4d -xvU' check failure.
SDP-387 (F): Test harness should run verify_sdp.sh and look for errors.
SDP-390 (B): Removed crontab refs to deleted scripts.
SDP-391 (B): In mkdir.sh, fixed root ownership of generated crontab file.

Released: Rev. SDP/MultiArch/2019.1/25386 (2019/03/21).

Jobs Fixed since change 25374 up to 25386 (F=Feature, B=Bug):

SDP-379 (B): Fixed mkdirs.sh issue with SSL-enabled installs due to chown issue.

Released: Rev. SDP/MultiArch/2019.1/25374 (2019/03/21).

Jobs Fixed since change 25320 up to 25374 (F=Feature, B=Bug):

SDP-302 (F): Parallelized checkpoint processing to reduce duration.
SDP-375 (B): Fix ownership and missing symlink issues in mkdirs.sh.

Released: Rev. SDP/MultiArch/2019.1/25320 (2019/03/08).

Jobs Fixed since change 25309 up to 25320 (F=Feature, B=Bug):

SDP-372 (B): Fixed breaking test - logic around $CN was misunderstood.

Released: Rev. SDP/MultiArch/2019.1/25309 (2019/03/07).

Jobs Fixed since change 25276 up to 25309 (F=Feature, B=Bug):

SDP-370 (B): Fix issue with mkdirs.sh -test mode.
SDP-371 (B): Fixed install_sdp_perl to work with updated P4Perl.

Released: Rev. SDP/MultiArch/2019.1/25276 (2019/03/06).

Jobs Fixed since change 25238 up to 25276 (F=Feature, B=Bug):

SDP-368 (B): Fix syntax errors for SDP_ADMIN_PASSWORD_FILE in backup_functions.sh.

Released: Rev. SDP/MultiArch/2019.1/25238 (2019/03/02).

Jobs Fixed since change 23583 up to 25238 (F=Feature, B=Bug):

SDP-198 (B): DOC: SDP Guide needs more detail on SDP-controlled replica setup.
SDP-247 (B): Need a validation script to check current status, especially after an upgrade from older version of SDP.
SDP-255 (B): Update SDP documentation to deal with SSL enabled servers.
SDP-256 (B): Update documentation to include mkdirs.cfg instead of updating mkdirs.sh
SDP-269 (F): Optimize journalPrefix values for master, edge, standby, replica.
SDP-296 (B): Fixed bug where updates to os_tweaks.sh dropped support for RHEL 6.5.
SDP-297 (B): Fixed handling of unlaod depots in p4verify.sh
SDP-298 (B): Avoid checking for shelved files in spec depot.
SDP-299 (F): Enhanced HMS failover for standby/journalcopy replicas to use 'p4 admin end-journal'
SDP-300 (F): Set rpl.journalcopy.location=1 for p4d 2018.1+ servers.
SDP-301 (B): Crontab spamming emails if configurable track=1
SDP-304 (B): Fixed issues with hms status for SSL-enabled servers, etc.
SDP-305 (F): Enhancemnts in libcore.sh bail(), errmsg(), initlog(), and stoplog() functions.
SDP-306 (B): Moved cset of P4JOURNAL so it’s done only on masteror edge server.
SDP-307 (F): Changed hms remote spec for the Public Depot to be the default 'origin'.
SDP-308 (F): Set rpl.journalcopy.location=1 for 2018.1+ standby replicas.
SDP-311 (B): Avoid unnecessary removal of rdb.lbr in db swap.
SDP-313 (F): Batch of validate_sdp.sh enhancements.
SDP-314 (B): Fixed scary but otherwise harmless error in Cmd.pm
SDP-315 (F): Reformat protections table comment format during upgrades.
SDP-316 (F): Added script to rotate live journal due low disk space
SDP-317 (B): Fixed issue preventing failover from an edge servder.
SDP-318 (B): Removed HMS references to /usr/local/bin/bash.
SDP-319 (F): Use 'p4 monitor terminate' rather than 'kill' for idle pids.
SDP-325 (F): Enhanced rejectList setting for P4EXP.
SDP-326 (F): Enhanced rejectList setting for P4EXP.
SDP-327 (B): DOC: must I create /hx*?
SDP-328 (B): DOC: mismatch between setup instructions and setup script
SDP-329 (B): DOC: folder permissions
SDP-330 (B): DOC: to where should we copy the tgz file?
SDP-331 (B): DOC: lack of clarity on folder ownership.
SDP-332 (B): DOC: setting executable bit on p4 etc.
SDP-333 (B): DOC: reorder the create user/group sequence on Linux.
SDP-334 (B): DOC: no comment on mkdirs.N.cfg, referenced in the mkdirs.sh script.
SDP-335 (B): DOC: no comment on ssl certificate generation.
SDP-337 (B): The upgrade.sh script, used to update your p4/p4d executables and corresponding database, starts with the message:
SDP-338 (F): DOC: server upgrades and checkpoint
SDP-339 (B): DOC: non-existent script referenced in docs
SDP-341 (B): Critical recreate_db_checkpoint.sh bug with shared /hxdepots shared.
SDP-342 (B): Fixed issue with P4Triggers.py impacting CheckCaseTrigger.py, etc.
SDP-345 (B): Implemented '-r' flag to clean up old logs in global_replica_status.sh.
SDP-347 (F): Enhance CaseCheckTrigger to work even if defaultChangeType is set to restricted.
SDP-348 (F): Enhanced CheckCaseTrigger.py to enable bypass.
SDP-350 (F): Support SELinux ("Security Enhanced Linux)") in enforcing mode.
SDP-353 (F): Avoid doing 'p4d -cset' for P4JOURNAL, but preserve safety feature.
SDP-362 (B): p4verify.sh -remote ignores SDP_RECENT_CHANGES_TO_VERIFY.
SDP-366 (F): Optimize display of Support messaging.
SDP-367 (B): p4verify.sh should also specify the -t (transfer) option for edge servers like it currently does for replicas.
SDP-49 (B): SDP Guide: inconsistent user name used in setup instructions. Page 9 of the pdf, step 1 says create user 'p4admin'. The rest of the guide that I’ve read, and the mkdirs.sh script, all refer to user 'perforce'.
SDP-73 (F): Add Evil Twin detector/prevention trigger.

Released: Rev. SDP/MultiArch/2018.1/23583.p2 (2019/01/23).

Jobs Fixed since change 23583.p1 up to 23583.p2 (F=Feature, B=Bug): * SDP-366 (F): Optimize display of Support messaging. * SDP-341 (B): Critical recreate_db_sync_replica.sh bug with shared /hxdepots shared.

Released: Rev. SDP/MultiArch/2018.1/23583.p1 (2018/07/13).

PATCH release fixes * SDP-341. * SDP-341 (B): Critical recreate_db_checkpoint.sh bug with shared /hxdepots shared.

Released: Rev. SDP/MultiArch/2018.1/23583 (2018/02/08).

Jobs Fixed since change 23504 up to 23583 (F=Feature, B=Bug):

SDP-287 (B): Fixed mkdirs.sh bug fixing ownership of wrong dir. 2nd fx for * SDP-287.

Released: Rev. SDP/MultiArch/2018.1/23504 (2018/01/19).

Jobs Fixed since change 23354 up to 23504 (F=Feature, B=Bug):

SDP-285 (B): The use of "tee" in p4d_base causes start to hang on some systems with 2016.2 p4d
SDP-286 (B): Fixed: In configure_new_server.*, super user not logged in after restart.
SDP-287 (B): Fixed mkdirs.sh bug fixing ownership of wrong dir.
SDP-290 (F): Added SDP-standard Ruby to PATH.
SDP-291 (F): Added workflow support triggers.
SDP-292 (F): Enhancements to p4dstate.sh.

Released: Rev. SDP/MultiArch/2017.4/23354 (2017/12/08).

Jobs Fixed since change 23329 up to 23354 (F=Feature, B=Bug):

SDP-281 (B): Fixed journalPrefix logic in mkdirs.sh

Released: Rev. SDP/MultiArch/2017.4/23329 (2017/12/05).

Jobs Fixed since change 23041 up to 23329 (F=Feature, B=Bug):

SDP-265 (B): sync_replica.sh and recreate_db_sync_replica.sh should avoid rsync compression
SDP-266 (B): Fixed logging issue with sync_replica.sh and recreate_db_sync_replica.sh.
SDP-267 (B): Fixed issue detecting an edge server; wrong EDGESERVER value.
SDP-268 (F): Added basic comments to all functions in backup_functions.sh.
SDP-269 (F): Optimize journalPrefix values for master, edge, standby, replica.
SDP-270 (F): Added trigger script to create Swarm reviews (and unit test).
SDP-271 (F): Refactor to push common arg_parse into P4Triggers.
SDP-272 (F): Enhance p4login info in error output, added -d debug flag.
SDP-273 (F): New RequireJob pre-submit trigger.
SDP-274 (F): Added trigger to validate contents YAML files.
SDP-276 (B): Make mkrep.sh work when /p4/hms instance does not exist.
SDP-277 (F): Added license time remaining check script.
SDP-278 (F): Enhance upgrade.sh run on machines with only broker/proxy.
SDP-279 (F): Added filetype.bypasslock=1 configurable, recommended for Swarm.
SDP-280 (F): Updated Support coverage info and enhanced other info in SDP README file.

Released: Rev. SDP/MultiArch/2017.3/23041 (2017/10/24).

Jobs Fixed since change 23003 up to 23041 (F=Feature, B=Bug):

SDP-143 (F): Enhance init scripts to display usage with no args.
SDP-259 (B): Fixed logging issue in upgrade.sh - p4broker/p4p start/stop not captured.
SDP-260 (F): Simplified stop_p4d() in backup function due to logic now in p4d_base.
SDP-261 (B): Fixed minor issue message about rotating journal escaping log capture.
SDP-262 (F): Take advantage of '--daemonsafe' arg in P4D 2017.1.
SDP-263 (B): Fixed login count in p4login script.

Released: Rev. SDP/MultiArch/2017.3/23003 (2017/10/19).

Jobs Fixed since change 22201 up to 23003 (F=Feature, B=Bug):

SDP-223 (B): Fix EDGESERVER check in p4d_base regression.
SDP-224 (B): Fix os_tweaks.sh to work on RHEL/CentOS 6.9.
SDP-225 (B): Updated hms_ts_setup.sh to adapt to mkdirs.cfg changes.
SDP-226 (B): Fixed typo in variable name in libcore.sh.
SDP-227 (B): install_sdp_perl.sh: Added DVCS environment isolation safety feature.
SDP-228 (F): install_sdp_perl.sh: Now uses r17.1 API.
SDP-232 (F): Re-establish autologinprompt default behavior.
SDP-233 (B): Fix bug in sdp_sync.sh related to args passed to rrun function.
SDP-234 (B): Change Unix default server.id to be unique to support remote depots
SDP-236 (B): Avoid issues interacting with incompatible local Perl.
SDP-237 (B): Scripts using libcore.sh display a harmless but scary error after all processing completes.
SDP-239 (F): Renamed crontab templates files to be template.job.p4s rather than *.template.
SDP-240 (B): Fixed cosmetic logging bug affecting several scripts.
SDP-241 (B): Update systemd templates to add new wait targets.
SDP-242 (B): Corrected filetypes on some scripts to chmod +x.
SDP-248 (F): Provide an override for diff between proxy listen and target ports.
SDP-249 (F): Extended P4IGNORE for more Mac files.
SDP-250 (F): Add value for db.monitor.shared configurable.
SDP-251 (F): Tweaked versions for which net.tcpsize=0 is set.
SDP-252 (B): SetWsOptionsAndView.py trigger should exit 0 when p4 client used w/ -t <template>.
SDP-253 (F): Various Windows SDP enhancements.
SDP-254 (F): Added SDP Overview slide preso to SDP.

Released: Rev. SDP/MultiArch/2017.2/22201 (2017/05/18).

Jobs Fixed since change 22177 up to 22201 (F=Feature, B=Bug):

SDP-222 (B): Fixed issue with testing mkdirs.sh on Mac.

Released: Rev. SDP/MultiArch/2017.2/22177 (2017/05/17).

Jobs Fixed since change 21808 up to 22177 (F=Feature, B=Bug):

SDP-153 (B): Maintenance script unload_clients_with_delete.py needs to ignore swarm workspaces to avoid losing data.
SDP-195 (F): Added new configurable setting: 'filesys.bufsize=1M'.
SDP-196 (B): Fixed bug in unload_labels.py, weeks passed in as days.
SDP-197 (B): Fixed bug in Windows Powershell p4verify.ps1 script.
SDP-200 (B): Fixed issue with /tmp getting polluted with temp files.
SDP-201 (F): Various scripts favor more scalable run()/rrun() over
SDP-202 (F): HMS scripts normalize to /bin/bash, check BASH_VERSION
SDP-203 (F): Added new JobIncrement trigger to simplify custom job name usage.
SDP-210 (F): Enhanced P4DTG init script reliability.
SDP-215 (F): Added new configurable setting: 'dm.user.loginattempts=7'.
SDP-216 (F): Refactored mkdirs.sh extracting user-config into mkdirs.cfg.
SDP-219 (B): Corrected logic on skipping the swarm user in remove_empty_pending_changes.py
SDP-220 (F): Added triggers using alternative file transfer for submit/pull.
SDP-221 (F): Enhanced to rotate p4verify.log like other logs.

Released: Rev. SDP/MultiArch/2017.1/21808 (2017/03/08).

Jobs Fixed since change 21720 up to 21808 (F=Feature, B=Bug):

SDP-191 (B): Fixed default threshholds values in proxy cache cleaner.
SDP-192 (B): User deletion maint script handles usernames with backslash.
SDP-193 (F): Enhance proxy init script to incorporate monitoring, adding default value in p4_vars.

Released: Rev. SDP/MultiArch/2017.1/21720 (2017/02/17).

Jobs Fixed since change 21528 up to 21720 (F=Feature, B=Bug):

SDP-179 (F): Enhanced install_sdp_perl.sh to build with P4Perl from The Workshop
SDP-181 (B): Fixed issue keeping recreate_db_sync_replca.sh from fixing a replica out-of-sync issue.
SDP-182 (F): Enhanced and updated install_sdp_perl.sh
SDP-183 (B): In recreate_db_sync_replica.sh, ensure P4ROOT and OFFLINE_DB are cleaned.
SDP-184 (B): Fixed bug resulting in error sending email on Ubuntu.
SDP-185 (F): Added PowerShell script p4verify.ps1 to verify on Windows.
SDP-186 (F): Simplified Windows setup with batch wrappers to PowerShell scripts.
SDP-187 (F): General Unix doc enhancements.
SDP-188 (F): Enhanced Perl library modules. SDP0190 (B): Fixed hms failover bug targeting wrong host with multiple possible targets.

Released: Rev. SDP/MultiArch/2016.2/21528 (2017/01/14).

Jobs Fixed since change 21489 up to 21528 (F=Feature, B=Bug):

SDP-160 (F): Enhanced P4Triggers.py to print exceptions encountered.
SDP-175 (F): Added more exclusions to /p4/.pignore.
SDP-176 (B): Fixed issues with hms_ts_setup.sh.
SDP-177 (B): Backed out clever tempdir feature that negatively impacted portability.

Released: Rev. SDP/MultiArch/2016.2/21489 (2017/01/11).

Jobs Fixed since change 21480 up to 21489 (F=Feature, B=Bug):

SDP-174 (B): Added Suffix field to spec depot in new servers.

Released: Rev. SDP/MultiArch/2016.2/21480 (2017/01/11).

Jobs Fixed since change 21379 up to 21480 (F=Feature, B=Bug):

SDP-167 (F): Added server.start.unlicensed configurable.
SDP-168 (F): Work around RHEL/CentOS 7 command completion issue.
SDP-169 (B): Fixed install_sdp_python.sh issue with building P4Python for SSL-enabled server.
SDP-170 (F): Added script to lowercase a directory tree.
SDP-171 (F): Added SSL support in hms_ts_setup.sh.
SDP-172 (B): Fixed bug in upgrade.sh introduced when truncate journal was changed to use p4 admin journal.
SDP-173 (F): Enhanced SSL support in install_sdp_python.sh to account for varied OpenSSL installations.

Released: Rev. SDP/MultiArch/2016.2/21379 (2016/12/20).

Jobs Fixed since change 21328 up to 21379 (F=Feature, B=Bug):

SDP-165 (F): Simplified SSL autogen cert generation.
SDP-166 (B): Re-login after setting auth.id, restart at end of script.

Released: Rev. SDP/MultiArch/2016.2/21328 (2016/12/16).

Jobs Fixed since change 21239 up to 21328 (F=Feature, B=Bug):

SDP-157 (F): Added sample /Users/perforce/.mailrc.
SDP-158 (F): SDP-ified log, moving p4triggers.log to ${LOGS}.
SDP-159 (F): Updated to hms tight ship setup script, still WIP.
SDP-161 (F): Added script to lowercase a dir tree.
SDP-162 (F): Added initial HMS tight ship script.
SDP-163 (F): Standardized doc format to docx.
SDP-164 (F): Enhanced to support enterprise environments with two metadata volumes.

Released: Rev. SDP/MultiArch/2016.2/21239 (2016/12/06).

Jobs Fixed since change 21193 up to 21239 (F=Feature, B=Bug):

SDP-154 (F): Enable auth.id for P4D 2016.1+ servers.
SDP-155 (F): Suppress known-harmless errors in p4verify.sh.
SDP-156 (F): Enhance install_sdp_p* scripts to remove hard-coded dependency on an instance 1.

Released: Rev. SDP/MultiArch/2016.2/21193 (2016/12/2).

SDP-138 (F): Take advantage of new 2016.2 configurables.
SDP-147 (F): Add CentOS 7 support to os_tweaks.sh
SDP-148 (F): Update SDP Python/Perl install to use 'http:' rather than 'ftp:' in URLs.
SDP-150 (B): Correct cp path for ssl dir.
SDP-151 (F): Enhance Windows SDP install to server.depot.root.
SDP-152 (F): Enhance to support 2 volumes for root/offline_db. Modified directory structure and scripts to support storing root and offline database on two different volumes. This is actually a preferred configuration now for performance and safety.

Released: Rev. SDP/MultiArch/2016.2/21123 (2016/11/22).

Jobs Fixed since change 21103 up to 21123 (F=Feature, B=Bug):

SDP-149 (B): Fix shebang line for case check trigger.
SDP-150 (B): Correct cp path for ssl dir.

Released: Rev. SDP/MultiArch/2016.2/21103 (2016/11/21).

Jobs Fixed since change 21101 up to 21103 (F=Feature, B=Bug):

SDP-146 (F): Added doc-only hms setup scripts.

Released: Rev. SDP/MultiArch/2016.2/21101 (2016/11/21).

Jobs Fixed since change 21033 up to 21101 (F=Feature, B=Bug):

SDP-144 (F): Add CheckCaseTrigger.p to the SDP.
SDP-145 (F): Various doc update for hms and failover.

Released: Rev. SDP/MultiArch/2016.2/21033 (2016/11/10).

Jobs Fixed since change 20995 up to 21033 (F=Feature, B=Bug):

SDP-139 (B): mkrep.sh: Add missing journalcopy to standby replicas.
SDP-140 (B): hms: Fixed issues with script hang on local failover.
SDP-141 (F): hms: Replaced hard-coded master ServerID value with configured value.
SDP-142 (B): Fix p4verify.sh bug when -recent flag is used.

Released: Rev. SDP/MultiArch/2016.2/20995 (2016/11/07).

Jobs Fixed since change 20972 up to 20995 (F=Feature, B=Bug):

SDP-134 (B): p4verify.sh is currently broken - use of an unbounded variable ($d); use of the -recent option fails as for verify -S you cannot specify a range (only @= is supported) ; stripping out archive,unload+remote depots also not working. get_verofy_rev_range used depot name incorrectly in changes
SDP-136 (B): Tweaked maint scripts to handle oddly named groups.
SDP-137 (F): Added new CN setting for /p4/common to test folder paths.

Released: Rev. SDP/MultiArch/2016.2/20972 (2016/11/01).

Jobs Fixed since change 20856 up to 20970 (F=Feature, B=Bug):

SDP-120 (F): Add -recent flag to p4verify.sh to verify recent changes.
SDP-126 (B): p4login.*.log files are not being cleaned up by the standard clean-up mechanism (remove_old_logs in backup_function.sh).
SDP-127 (F): Allow for multiple depot volumes.
SDP-128 (B): Fixed missing LOGFILE varialbe issue in p4verify.sh.
SDP-129 (B): Fix formatting of logs to use simple HTML line breaks for Windows SDP.
SDP-130 (B): Various test suite improvements.
SDP-131 (B): fixed issue with broken def’n of P4PORT with multiple SSL prefixes.
SDP-132 (F): Enhanced two-factor auth mechanims.
SDP-133 (F): Tweak to slightly reduce mkdirs.sh configuration.

Released: Rev. SDP/MultiArch/2016.2/20856 (2016/10/04).

Jobs Fixed since change 20805 up to 20856 (F=Feature, B=Bug):

SDP-121 (F): Enhanced logic to favor init scripts in /p4/N/bin over /etc/init.d.
SDP-122 (B): Fixed quoting bug in p4login.
SDP-123 (F): hms checks for cfg files in /p4/common/config.
SDP-124 (B): Fixed hms bug re: wrong master host discovered in failback testing

Released: Rev. SDP/MultiArch/2016.2/20805 (2016/10/02).

Jobs Fixed since change 20790 up to 20805 (F=Feature, B=Bug):

SDP-116 (B): Fixed ssh hang issues with hms scripts.
SDP-117 (F): Implemented global/central start stop.
SDP-118 (F): mkrep.sh: Added more replica types and journalcopy support.
SDP-119 (F): Implement checkauth feature in HMS broker_wrapper.

Released: Rev. SDP/MultiArch/2016.2/20790 (2016/09/30).

Jobs Fixed since change 20755 up to 20790 (F=Feature, B=Bug):

SDP-112 (B): Minor p4 login fix to show P4TICKETS output reliably.
SDP-113 (B): Fix mkrep.sh to use $P4BIN instead of p4.
SDP-114 (F): Added basic p4login-super.sh script.
SDP-115 (B): Added hms dir to mkdirs.sh.

Released: Rev. SDP/MultiArch/2016.2/20755 (2016/09/29).

Jobs Fixed since change 20559 up to 20755 (F=Feature, B=Bug):

SDP-64 (F): Add option to install_sdp_python.sh to build P4Python with SSL.
SDP-103 (B): Fix attempted login count in p4login.
SDP-104 (B): Fixed order-of-operations issue that caused some log files to be lost on rotation.
SDP-105 (B): Cosmetic fix to broker init script with 'status all' run w/no config files.
SDP-106 (F): Add Helix Management System (HMS) files to SDP.
SDP-107 (F): Enhance p4login to avoid hang issue with auth.id set.
SDP-108 (F): Further test suite docker environment updates.
SDP-109 (F): Add Two Factor Authentication sample auth script.
SDP-110 (B): Correct crontab to run sync_replica.sh every day.
SDP-111 (B): Fix bugs in convert_label_to_autoreload.py.

Jobs Fixed since change 20491 up to 20559 (F=Feature, B=Bug):

SDP-100 (B): Fix recent regression with 'restart' option in init scripts.

Released: Rev. SDP/MultiArch/2016.1/20491 (2016/09/09).

Jobs Fixed since change 20460 up to 20491 (F=Feature, B=Bug):

SDP-99 (B): Fix special case env insulation bug running init scripts as root.

Released: Rev. SDP/MultiArch/2016.1/20460 (2016/09/08).

Jobs Fixed since change 20395 up to 20460 (F=Feature, B=Bug):

SDP-94 (B): Fixed a typo inserted in mkdirs.sh.
SDP-98 (B): chmod +x fix on some scripts.

Released: Rev. SDP/MultiArch/2016.1/20395 (2016/09/04).

Jobs Fixed between changelists 20387 and 20395 (F=Feature, B=Bug):

SDP-94 (B): Fixed a typo inserted in mkdirs.sh.

Released: Rev. SDP/MultiArch/2016.1/20387 (2016/09/04).

Jobs Fixed between changelists 20348 and 20387 (F=Feature, B=Bug):

SDP-89 (B): Fixed several issues in email.sh maint script.
SDP-92 (B): Fixed using with runRemoteCmd getting error if USER not defined in non-core SDP script.
SDP-93 (B): Removed references to legacy names for checkpoint scripts (non-functional change).

Jobs Fixed between changelists 20028 and 20348 (F=Feature, B=Bug):

SDP-64 (F): Add option to install_sdp_python.sh to build P4Python with SSL.
SDP-74 (F): Add p4sanity_check.sh script from Battle School.
SDP-75 (B): Fix missing echo of command into log file in p4verify.sh.
SDP-77 (F): Change adminspass to use hidden, and now instance-specific, password files
SDP-78 (F): Add mkrep script to simplify replica creation.
SDP-79 (F): Update p4verify.sh to verify shelves on master/replicas.
SDP-80 (F): Add script to convert labels to autoreload.
SDP-81 (F): Add Evil Twin Detector maintenance script.
SDP-85 (F): Add script to simplify adoption of OS-level best practices recommended by Support.
SDP-97 (B): Eliminate syntax error if user forgets start/stop/status arg; just display status.
SDP-88 (F): Use pid to shutdown p4d rather than 'p4 admin stop' to enhance reliability.

Released: Rev. SDP/MultiArch/2016.1/20028 (2016/08/03).

Jobs Fixed between changelists 19888 and 20028 (F=Feature, B=Bug):

SDP-67 (F): Defend against environment hijacking of P4ENVIRO.
SDP-72 (F): Revive external auth LDAP group mirror scripts.
SDP-69 (B): sdp_sync.sh v2.1.7: Fixed environment bug (missing defs), HMS support.
SDP-70 (F): Simplify login ticket handling for sophisticated topologies.
SDP-71 (F): Prevent cross-instance contamination if p4_vars is loaded multiple times.

Released: Rev. SDP/MultiArch/2016.1/19888 (2016/07/07).

Jobs Fixed between changelists 19838 and 19888 (F=Feature, B=Bug):

SDP-6 (B): SDP trigger breaks with Streams workspaces in P4V.
SDP-66 (B): Daily/weekly_backup can fail leaving lock file present requiring manual intervention
SDP-67 (F): Defend against environment hijacking of P4ENVIRO.

Released: Rev. SDP/MultiArch/2016.1/19838 (2016/06/26).

Jobs Fixed between changelists 19768 and 19838 (F=Feature, B=Bug):

SDP-6 (B): SDP triggers break with Streams workspaces in P4V.
SDP-65 (F): Add defense against known-misbehaving client versions.

Released: Rev. SDP/MultiArch/2016.1/19768 (2016/06/24).

Jobs Fixed between changelists 19661 and 19768 (F=Feature, B=Bug):

SDP-62 (F): Change to best-practice configurables; auth.autologinprompt=0.
SDP-63 (B): p4master_run operates incorrectly as root.

Released: Rev. SDP/MultiArch/2016.1/19661 (2016/06/08).

Jobs Fixed between changelists 19410 and 19661 (F=Feature, B=Bug):

SDP-59 (F): Add KEEPJNLS to allow keeping more jnls than ckps.
SDP-60 (F): Rename replica scripts to be consistent with earlier rename of checkpoint scripts.
SDP-61 (F): Enhance error checkong and logging replica handling scripts.
SDP-62 (F): Change to best-practice configurables; auth.autologinprompt=0.

Released: SDP/MultiArch/2016.1/19410 (2016/05/17).

Jobs Fixed between changelists 18958 and 19410:

SDP-50: Work with license check in P4D 2016.1.
SDP-51: Change p4verify.sh to use -S to verify shelves on a replica instead
SDP-52: Removed the HOST_IP settings from mkdirs and instance_vars since it
SDP-53: Dockerify SDP test suite.
SDP-54: Add replica cleanup script on Windows.
SDP-55: Get p4verify.sh to check SHAREDDATA on replicas.
SDP-56: Avoid chmod on a replica if it shares data with the master.
SDP-57: Use 'p4 admin journal' on master to rotate journal.

Released: SDP/MultiArch/2016.1/18958 (2016/04/08).

Jobs Fixed between changelists 18619 and 18958:

SDP-39: Proxy status init script can show proxy is up when it is not.
SDP-41: Moved ckp_running.txt to LOGS dir.
SDP-42: Fix missing LOGFILE= in replica_cleanup.sh.
SDP-43: De-emphasized changeType=restricted.
SDP-44: Update Perl/P4Perl Python/P4Python
SDP-45: Support multiple broker configs.
SDP-46: Enhanced p4_vars.template to support non-p4d hosts.
SDP-47: SDP-ify P4DTG init script.
SDP-48: Enhanced p4deleteuser.py to check for shelves, double-deletes.

Released: SDP/MultiArch/2016.1/18619 (2016/03/10).

Jobs Fixed between changelists 17297 and 18619:

SDP-30 on 2015/12/15 by ttyler closed 'Enhanced offline_db 'stale data'

Jobs Fixed between changelists 16784 and 17297:

SDP-32: Support $MAILTO values with'#' char in gen_default_broker_cfg.sh.
SDP-34: sdp_sync.sh: Fix 'null crontab' bug.
SDP-35: p4d_base: When stopping p4d, send stderr to /dev/null.
SDP-36: global_replica_status.sh: Support instance-specifc host config.
SDP-37: Support email addresses starting with '#'.

Released: SDP/MultiArch/2016.1/16784 (2016/01/12).

Jobs Fixed recently:

SDP-31: Add documentation for sdp_sync.sh to SDP Guide.
SDP-32: Support $MAILTO values with'#' char in gen_default_broker_cfg.sh.