USAGE for v5.8.8: p4verify.sh [] [-nu] [-nr] [-ns] [-nS] [-a] [-nt] [-nz] [-o BAD|MISSING] [-chunks ] [-recent] [-dlf ] [-n] [-L ] [-v] [-d] [-D] or p4verify.sh -h|-man DESCRIPTION: This script performs a 'p4 verify' of all submitted and shelved versioned files in depots of all types except 'remote' and 'archive' type depots. If run on a replica, it schedules archive failures for transfer to the replica. OPTIONS: Specify the SDP instance. If not specified, the SDP_INSTANCE environment variable is used instead. If the instance is not defined by a parameter and SDP_INSTANCE is not defined, p4verify.sh exists immediately with an error message. -nu Specify '-nu' (No Unload) to skip verification of the singleton depot of type 'unload' (if created). The 'unload' depot is verified by default. -nr Specify '-nr' (No Regular) to skip verification of regular submitted archive files. The '-nr' option is not compatible with '-recent'. Regular submitted archive files are verified by default. -ns Specify '-ns' (No Spec Depot) to skip verification of singleton depot of type 'spec' (if created). The 'spec' depot is verified by default. -nS Specify '-nS' (No Shelves) to skip verification of shelved archive files, i.e. to skip the 'p4 verify -qS'. -a Specify '-a' (Archive Depots) to do verification of depots of type 'archive'. Depots of type 'archive' are not verified by default, as archive depots are often physically removed from the server's storage subsystem for long-term cold storage. -nt Specify the '-nt' option to avoid passing the '-t' flag to 'p4 verify' on a replica. By default, p4verify.sh detects if it is running on a replica, and if so automatically applies the '-t' flag to 'p4 verify'. That causes the replica to attempt to self-heal, as files that fail verification are scheduled for transfer from the P4TARGET server. This default behavior results in 'Transfer scheduled' messages in the log, and MISSING/BAD files are listed as 'info:' rather than 'error:'. There is no clear indication of whether or which of the scheduled transfers complete successfully, and so there may be a mix of transient/correctable and "real"/persistent transfer errors for files that are also BAD/MISSING on the master server. Specify '-nt' to ensure the log contains a list of files that currently fail a 'p4 verify' without attempting to transfer them from the master. -nz Specify '-nt' to skip the gzip of the old log file. By default, if a log with the default name or the name specified with '-L' exists at the start of processing, the old log is rotated and gzipped. With this option the old log is not zipped when rotated. -o BAD|MISSING Specify '-o MISSING' to check only whether expected archive files exist or not, skipping the checksum calculation of existing files. This results in dramatically faster, if less comprehensive, verification. This is particularly well suited when verification is being used to schedule archive file transfers of missing files on replicas. This translates into passing the '--only MISSING' option to 'p4 verify'. Specify '-o BAD' to check only for BAD revisions. This translates into passing the '--only BAD' option to 'p4 verify'. This option requires p4d to be 2021.1 or newer. For older p4d versions, this option is silently ignored. -chunks Specify the maximum amount of content by size to verify at once. If this is specified, the depot_verify_chunks.py script is used to break up depots into chunks of a given size, e.g. 100M or 4G. The parameter must be a size value valid to pass to the depot_verify_chunks.py script with the '-m' option. This requires the perforce-p4python3 module to be installed and the python3 in the PATH must be the correct one that uses the P4 module. The -recent and -chunks options are mutually exclusive. -recent Specify that only recent changelists should be verified. The $SDP_RECENT_CHANGES_TO_VERIFY variable defines how many changelists are considered recent; the default is 200. If the default is not appropriate for your site, add "export SDP_RECENT_CHANGES_TO_VERIFY" to /p4/common/config/p4_N.vars to change the default for an instance, or to /p4/common/bin/p4_vars to change it globally. If $SDP_RECENT_CHANGES_TO_VERIFY is unset, the default is 200. When -recent is used, neither shelves nor files in the unload depot are verified. -dlf Specify a file containing a list of depots to process in the desired order. By default, all depots reported by 'p4 depots', which effectively results in depots processed in alphabetical order. This can be useful in time-sensitive situations where the order of processing can be prioritized, and/or to prevent processing certain depots. The format fo the depot list file is straighforward, one line per depot, without the '//' nor trailling /..., so a list might look like this sample: ProjA ProjB spec .swarm unload archive ProjC Blank lines and lines starting with a '#' are treated as comments and ignored. WARNING: This is not intended to be the primary method of verification, because it would be easy to forget to add new depots to the list file. If the depot list file is not readable, processing aborts. -v Verbose. Show output of verify attempts, which is suppressed by default. Setting SDP_SHOW_LOG=1 in the shell environment has the same effect as -v. The default behavior of this script is to generate no terminal output, but instead to write output into a log file -- see LOGGING below. If '-v' is specified, the generated log is sent to stdout at the end of processing. This flag is not recommended for routine cron operation or for large data sets. The -chunks and -recent options are mutually exclusive. -L Specify the log file to use. The default is /p4/N/logs/p4verify.log Log rotation and old log cleanup logic does not apply to log files specified with -L. Thus, using -L is not recommended for routine scheduled operation, e.g. via crontab. DEBUGGING OPTIONS: -n No-Operation (NO_OP) mode, for debugging. Display certain commands that would be executed without executing them. When '-n' is used, commands that might take a long time to run or affect data are only displayed. Even in '-n' mode, some information-gathering commands such as listing shelved CLs are executed, which may cause the script to take a bit of time to run on a large data set even in dry run mode. -d Specify that debug messages should be displayed. -D Use bash 'set -x' extreme debugging verbosity, and imply '-d'. -L off The special value '-L off' disables logging. This can only be used with '-n' for debugging. HELP OPTIONS: -h Display short help message -man Display man-style help message EXAMPLES: Example 1: Full Verify This script is typically called via cron with only the instance parameter as an argument, e.g.: p4verify.sh 1 Example 2: Fast Verify A "fast" verify is one in which only the check for MISSING archives is done, while the resource-intensive checksum calculation of potentially BAD existing archives is skipped. This is especially useful when used on a replica. p4verify.sh 1 -o MISSING Example 3: Fast and Recent Verify The '-o MISSING' and '-recent' flags can be combined for a very fast check. This check might be incorporated into a failover procedure. p4verify.sh 1 -o MISSING -recent Example 4: Submitted Files Only This will verify only use submitted files, ignoring shelves and the spec and unload depots, putting the results in a specified log: p4verify.sh 1 -ns -nS -nu -L -L /p4/1/logs/p4verify.submitted.log Example 5: Shelved Files Only This will verify only use submitted files, ignoring shelves and the spec and unload depots, putting them in a specified log: p4verify.sh 1 -nr -ns -nu -L /p4/1/logs/p4verify.shelved.log Example 6: A Dry Run The '-n' option can be used for a dry run. Output may also be displayed to the screen ('-v') for a dry run and the log file optionally discarded: p4verify.sh 1 -n -nS -L off -v NOHUP USAGE: Because archive verification is typically a long running task, it is advisable to use 'nohup' to call each command, and combine that by running the command as a background process. Alternately, use 'screen' or similar. Any of the examples above can be used with 'nohup', without output redirected to /dev/null (i.e. to "the void", as this script handles logging and output redirection). To use 'nohup', start the command line with 'nohup', and then after the command, add this text exactly: < /dev/null > /dev/null 2>&1 & As a example, Example 2 above, called with nohup, would look like: nohup /p4/common/bin/p4verify.sh 1 -o MISSING < /dev/null > /dev/null 2>&1 & With the ampersand '&' at the end, the command will appear to return immediately as the process continues to run in the background. Then optionally monitor the log: tail -f /p4/1/logs/p4verify.log LOGGING: This script generates no output by default. All (stdout and stderr) is logged to /p4/N/logs/p4verify.log. The exception is usage errors, which result an error being sent to stderr followed usage info on stdout, followed by an immediate exit. NOTIFICATIONS: In addition to logging, a short summary of the verify is sent as a notification. The summary is reliably short even if the output of the verifications done by this script results in a large log file. There are two notification schemes with this script: * Email notification is always attempted. * AWS SNS notification is attempted if the SNS_ALERT_TOPIC_ARN custom setting is defined. This is typically set in: /p4/common/site/config/p4_N.vars.local EXIT CODES: An exit code of 0 indicates no errors were encountered attempting to perform verifications, AND that all verifications attempted reported no problems. A exit status of 1 indicates that verifications could not be attempted for some reason. A exit status of 2 indicates that verifications were successfully performed, but that problems such as BAD or MISSING files were detected, or else system limits prevented verification.