externalcopy.txt #1

Using alternative file transfer from submit and pull in distributed environment.

Requirements:
submit from an edge server uses alternative file transfer from edge to
  commit server using an edge-content trigger.
pull from an edge server uses alternative file transfer using 
  pull-archive triggers.


Setup for submit.
p4 configure set lbr.autocompress=1
p4 configure set rpl.submit.nocopy=1
p4 configure set pull.trigger.dir=<location to write temp files>

A user must be set up with the public keys and anything else needed
  to perform the copy. In the case of using aspera, this would require
  ssl public keys or the path to a key, and a license or you set the
  Aspera passord in the environment.

edge-content trigger: this will invoke the copy commands using a
   list of files generated from its 'fstat -Ob' command.
   For the fstat command, it needs a user with admin privilege.
   For the copy command, it needs the user, target host, target path,
   source path. The changelist must be passed as an argument to the
   trigger, as well as any of the other above-mentioned parameters
   if they are not hard-coded in the trigger script.
  The trigger variable %triggerdir% has the value of pull.trigger.dir.

How this works:
  The edge-content trigger invokes 'p4 fstat -Ob @=<shelvedChange>'
  in order to get a list of files to transfer. In addition, it filters
  out any lazy copies since those don't require transfer.
  It will invoke the copy command (in this case ascp) to transfer the
  files from the shelved change from the edge server to the commit server.
  If a file transfer failure is detected from the trigger, the error returned
  will cause the submit to fail.

  Upon success the submit continues. When rmtSubmitShelf is run, it
  will Stat() the files on the commit server to verify that the transfer
  occurred instead of performing the transfer.
  If any files are not found, an error is returned from submit.
  
Setup for pull.
p4 configure set lbr.autocompress=1
p4 configure set pull.trigger.dir=<location to write temp files>
p4 configure set lbr.replica.notransfer=1

pull-archive trigger: This will invoke the copy command by reading the list of
  files from the file whose filename was passed as a parameter to the trigger
  script.  For the copy command, it needs the user, target host, target path,
  source path. 
  The trigger variable %triggerdir% has the value of pull.trigger.dir.
  The trigger variable %archiveList% has the name of the file containing
  the files to be copied.

How this works:
  There may be several startup.* configurables using 'pull --trigger'.
  When file content should be pulled, each pull will try
  to grab a batch of work. Then it goes through the list of files to
  be pulled, creates a temp file and writes the files to be transfered
  to the temp file. If any min-size or max-size parameters are specified,
  the pull will use them to filter out files not meeting the constraint.
  It then invokes the pull-archive trigger with the file containing its
  file list.  The pull-archive trigger invokes the copy command
  (in this case ascp) to transfer the files from the commit to the edge.
  Transfer errors are reported to the log file when rpl=4. Files that
  failed to transfer will be detected and transferred by regular pull
  threads (not --trigger). Files are checked for transfer error by
  using Stat() unless the trigger script reported an error. If the
  trigger reported an error, then the checksums are compared to 
  determine if transfer failed.

  When lbr.replica.notransfer is configured, sync commands accessing files
  being pulled by the pull-archive triggers will wait for the pull transfer
  to finish rather than initiating another fetch mechanism.