#!/bin/bash #------------------------------------------------------------------------------ # Copyright and license info is available in the LICENSE file included with # the Server Deployment Package (SDP), and also available online: # https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/LICENSE #------------------------------------------------------------------------------ # This is a reference (example) pull-archive trigger for use with an Edge/Commit server topology - the # Edge server uses this trigger to pull files from its upstream Commit server, # using Aspera for fast file transfer (rather than builtin pull threads). # Aspera uses UDP rather than TCP and is typically much faster, especially with high latency # connections. # See: https://community.perforce.com/s/article/15337 #------------------------------------------------------------------------------ # Read filename to get list of files to copy from commit to edge. # Do the copy using ascp (Aspera file copy) # # configurable pull.trigger.dir should be set to a temp folder like /p4/1/tmp # # Startup commands look like: # startup.2=pull -i 1 -u --trigger --batch=1000 # # The trigger entry for the pull commands looks like this: # # pull_archive pull-archive pull "/p4/common/bin/triggers/pull.sh %archiveList%" # # There are some pull trigger options, but the are not necessary with Aspera. # Aspera works best if you give it the max batch size of 1000 and set up 1 or more # threads. Note, that each thread will use the max bandwidth you specify, so a # single pull --trigger thread is probably all you will want. # # The ascp user needs to have ssl public keys set up or export ASPERA_SCP_PASS. # The ascp user should be set up with the target as / with full write access to the volume where # the depot files are located. The easiest way to do that is to use the same user that is # running the p4d service. # # WARNING - ensure ascp is correctly configured and working in your environment: # https://www-01.ibm.com/support/docview.wss?uid=ibm10747281 # (search for "ascp connectivity testing") # # Standard SDP environment is assumed, e.g P4USER, P4PORT, OSUSER, P4BIN, etc. # are set, PATH is appropriate, and a super user is logged in with a non-expiring # ticket. set -u function msg () { echo -e "$*"; } function bail () { msg "\nError: ${1:-Unknown Error}\n"; exit ${2:-1}; } [[ $# -eq 1 ]] || bail "Bad Usage!\n\nUsage:\n\t$0 %archiveList%\n\n" LOGFILE=$LOGS/pull.log fileList=$1 date=$(date '+%Y-%m-%d %H:%M:%S') echo "$date $fileList" >> $LOGFILE # Set this value appropriately, or to ensure it is not accidentally checked in, # set it in the appropriate SDP file, e.g. /p4/common/config/p4_1.vars export ASPERA_SCP_PASS=xxxx [[ -f $fileList ]] || { echo "$fileList missing!" >> $LOGFILE; exit 1; } # Note: the -l parameter should be adjusted to how much bandwidth you want Aspera to use # in your environment. # Optional for debugging. See above for checking ascp connectivity # echo "ascp -P 33001 -l 500M -d --mode=recv --file-pair-list=$fileList --user=$OSUSER --host=$P4MASTER --partial-file-suffix=asp /"" >> $LOGFILE ascp -P 33001 -l 500M -d --mode=recv --file-pair-list=$fileList --user=$OSUSER --host=$P4MASTER --partial-file-suffix=asp / >> $LOGFILE 2>&1 if [[ $? -ne 0 ]]; then bail "$0: failed to execute the ascp command, contact Perforce admin.\n" fi exit 0
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#7 | 29923 | C. Thomas Tyler |
Updated HTML hyperlinks to use 'portal.perforce.com'. This replaces currently broken links to 'answers.perforce.com' and currently redirected links to 'community.perforce.com'. #review-29924 |
||
#6 | 26851 | Robert Cowham |
Fix typo in tmpfs /etc/fstab entry which stopped it working in the doc. Mention in pre-requisites for failover and failover guide the need to review OS Config for your failover server. Document Ubuntu 2020.04 LTS and CentOS/RHEL 8 support. Note performance has been observed to be better with CentOS. Document pull.sh and submit.sh in main SDP guide (remove from Unsupported doc). Update comments in triggers to reflect that they are reference implementations, not just examples. No code change. |
||
#5 | 26773 | Robert Cowham |
Move Aspera triggers back to supported section as referred to in KB and Sysadmin Guide. Clarify description in file headers and link to KB article. |
||
#4 | 26652 | Robert Cowham |
This is Tom's change: Introduced new 'Unsupported' directory to clarify that some files in the SDP are not officially supported. These files are samples for illustration, to provide examples, or are deprecated but not yet ready for removal from the package. The Maintenance and many SDP triggers have been moved under here, along with other SDP scripts and triggers. Added comments to p4_vars indicating that it should not be edited directly. Added reference to an optional site_global_vars file that, if it exists, will be sourced to provide global user settings without needing to edit p4_vars. As an exception to the refactoring, the totalusers.py Maintenance script will be moved to indicate that it is supported. Removed settings to support long-sunset P4Web from supported structure. Structure under new .../Unsupported folder is: Samples/bin Sample scripts. Samples/triggers Sample trigger scripts. Samples/triggers/tests Sample trigger script tests. Samples/broker Sample broker filter scripts. Deprecated/triggers Deprecated triggers. To Do in a subsequent change: Make corresponding doc changes. |
||
#3 | 26048 | Robert Cowham |
Added doc notes for pull.sh and links to details. Added pull_test.sh for easy local testing of the config. |
||
#2 | 22616 | Russell C. Jackson (Rusty) | Updated comments about number of threads. | ||
#1 | 22107 | Russell C. Jackson (Rusty) | External copy scripts and notes. | ||
//guest/perforce_software/sdp/dev/Unsupported/Samples/triggers/pull.sh | |||||
#1 | 26652 | Robert Cowham |
This is Tom's change: Introduced new 'Unsupported' directory to clarify that some files in the SDP are not officially supported. These files are samples for illustration, to provide examples, or are deprecated but not yet ready for removal from the package. The Maintenance and many SDP triggers have been moved under here, along with other SDP scripts and triggers. Added comments to p4_vars indicating that it should not be edited directly. Added reference to an optional site_global_vars file that, if it exists, will be sourced to provide global user settings without needing to edit p4_vars. As an exception to the refactoring, the totalusers.py Maintenance script will be moved to indicate that it is supported. Removed settings to support long-sunset P4Web from supported structure. Structure under new .../Unsupported folder is: Samples/bin Sample scripts. Samples/triggers Sample trigger scripts. Samples/triggers/tests Sample trigger script tests. Samples/broker Sample broker filter scripts. Deprecated/triggers Deprecated triggers. To Do in a subsequent change: Make corresponding doc changes. |
||
# | Change | User | Description | Committed | |
#3 | 26048 | Robert Cowham |
Added doc notes for pull.sh and links to details. Added pull_test.sh for easy local testing of the config. |
||
#2 | 22616 | Russell C. Jackson (Rusty) | Updated comments about number of threads. | ||
#1 | 22107 | Russell C. Jackson (Rusty) | External copy scripts and notes. |