#!/bin/bash #============================================================================== # Copyright and license info is available in the LICENSE file included with # the Server Deployment Package (SDP), and also available online: # https://swarm.workshop.perforce.com/projects/perforce-software-sdp/view/main/LICENSE #------------------------------------------------------------------------------ # This script is designed to create a seed checkpoint for an Edge server. # # An edge server is naturally filtered, with certain database tables (e.g. # db.have) excluded. In addition to implicit filtering, the server spec may # specify additional tables to be excluded, e.g. by using the # ArchiveDataFilter field of the server spec. # # The script requires the SDP instance and the edge ServerID. # Run example: # ./edge_dump.sh 1 p4d_edge_syd function usage () { echo -e "Usage:\n\t${0##*/} <SDP_Instance> <ServerID>\n" exit 1 } [[ $# -ne 2 || ${1:-Unset} == -h ]] && usage export SDP_INSTANCE=${SDP_INSTANCE:-Undefined} export SDP_INSTANCE=${1:-$SDP_INSTANCE} if [[ $SDP_INSTANCE == Undefined ]]; then echo -e "Usage Error: Instance parameter not supplied." usage fi declare ServerID=${2:-Unset} if [[ $ServerID == Unset ]]; then echo -e "Usage Error: ServerID parameter not supplied." usage fi source /p4/common/bin/p4_vars $SDP_INSTANCE source /p4/common/bin/backup_functions.sh LOGFILE="$LOGS/edge_dump.$(date +'%Y%m%d-%H%M').log" declare EdgeSeedCheckpoint= declare ExcludedTables= declare DumpCmd= ######### Start of Script ########## echo "Processing. This may take a while depending on checkpoint duration." echo "Log file is: $LOGFILE" check_vars set_vars check_offline_db_usable get_offline_journal_num if [[ "$(is_edge $ServerID $OFFLINE_DB)" == YES ]]; then log "Verified: Server $ServerID is an edge server." else die "Could not confirm that $ServerID is an edge server." fi ckp_running ExcludedTables=db.have,db.working,db.resolve,db.locks,db.revsh,db.workingx,db.resolvex EdgeSeedCheckpoint=${CHECKPOINTS}/${P4SERVER}.${ServerID#p4d_}.seed.$OFFLINEJNLNUM.gz DumpCmd="$P4DBIN -r $OFFLINE_DB -K $ExcludedTables -P $ServerID -jd -z $EdgeSeedCheckpoint" log "Create edge seed checkpoint from master offline_db skipping edge-local tables and using the edge server form for any filtered data." log "Running: $DumpCmd" { time $DumpCmd ; } >> $LOGFILE 2>&1 || { die "Edge seed checkpoint dump failed!"; } log "Edge seed checkpoint complete." # Inform the user about follow up tasks log "You now need to copy $EdgeSeedCheckpoint to the edge server and use /p4/common/bin/recover_edge.sh to reseed the edge server." ckp_complete cat $LOGFILE
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#1 | 23960 | noe_gonzalez | "Forking branch Dev of perforce-software-sdp to noe_gonzalez-sdp." | ||
//guest/perforce_software/sdp/dev/Server/Unix/p4/common/bin/edge_dump.sh | |||||
#7 | 23297 | C. Thomas Tyler |
Added safety checks to avoid running commands that will certainly fail in upgrade.sh. Generally, /p4/common/bin will be the same on all hosts in a Helix topolgy. However, on any given machine, the /p4/<N>/bin/<EXE>_<N>_init scripts should exist only for executables that run on that machine. This change to upgrade.sh should work on machines even where only a proxy or broker runs. Also, it will not generate errors in cases where there is, say, a p4p_N_bin symlink in /p4/common/bin but no /p4/N/bin/p4p_N_init script, which will a common situation since /p4/common/bin will contain all executables used anywhere, while /p4/N/bin is host-specific. Also made cosmetic fixes and style convergence change. In dump_edge.sh and recover_edge_dump.sh, just fixed cosmetic typos. |
||
#6 | 23266 | C. Thomas Tyler |
Fixes and Enhancements: * Enabled daily_checkpoint.sh operate on edge servers, to keep /p4/N/offline_db current on those hosts for site-local recovery w/o requiring a site-local replica (though having a site-local replica can still be useful). * Disabled live_checkpoint.sh for edge servers. * More fully support topologies using edge severs, in both geographically distributed and horizaontal scaling "wokspace server" solutions. * Fix broken EDGESERVER value definition. * Modified name of SDP counter that gets set when a checkpoint is taken to incorporate ServerID, so now the counter name will look like lastSDPCheckpoint.master.1, or lastSDPCheckpoint.p4d_edge_sfo, rather than just lastSDPCheckpoint. There will be multiple such counters in a topology that uses edge servers, and/or which takes checkpoints on replicas. * Added comments for all functions. For the master server, journalPrefix remains: /p4/N/checkpoints/p4_N The /p4/N/checkpoints is reserved for writing by the master/commit server only. For non-standby (possibly filtered) replicas and edge serves, journalPrefix is: /p4/N/checkpoints.<ShortServerID>/p4_N.<ShortServerID> Here, ShortServerID is just the ServerID with the 'p4d_' prefix trimmed, since it is redundant in this context. See mkrep.sh, which enshines a ServerID (server spec) naming standard, with values like 'p4d_fr_bos' (forwarding replica in Boston) and p4d_edge_blr (Edge server in Bangalore). So the journalPrefix for the p4d_edge_bos replica would be: /p4/N/checkpoints.edge_bos/p4_N.edge_bos For "standby" (aka journalcopy) replicas, journalPrefix is set to /p4/N/journals.rep. which is written to the $LOGS volume, due to the nature of standby replicas using journalPrefix to write active server logs to pre-rotated journals. Some take-away to be updated in docs: * The /p4/N/checkpoints folder must be reserved for checkpoints that originate on the master. It should be safe to rsync this folder (with --delete if desired) to any replica or edge server. This is consistent with the current SDP. * I want to change 'journals.rep' to 'checkpoints.<ShortServerID>' for non-standby replicas, to ensure that checkpoints and journals taken on those hosts are written to a volume where they are backed up. * In sites with multiple edge serves, some sharing achive files ('workspace servers'), multiple edge servers will share the same SAN. So we one checkpoints dir per ServerID, and we want that dir to be on the /hxdepots volume. Note that the journalPrefix for replicas was a fixed /p4/N/journals.rep. This was on the /hxlogs volume - a presumably fast-for-writes volume, but typically NOT backed up and not very large. This change puts it under /p4/N/checkpoints.* for edge servers and non-standby replicas, but ensures other replica types and edge servers can generate checkpoints to a location that is backed up and has plenty of storage capacity. For standby replicas only (which cannot be filtered), the journalPrefix remains /p4/N/journals.rep on the /hxlogs volume. |
||
#5 | 22752 | Russell C. Jackson (Rusty) | Corrected comments about -K flag. | ||
#4 | 22720 | Robert Cowham | Fix script to create a filtered checkpoint for edge server. | ||
#3 | 22617 | Russell C. Jackson (Rusty) | Added +x | ||
#2 | 22615 | Russell C. Jackson (Rusty) |
Added suggested comments. By passing review since this is a comment only change. |
||
#1 | 22614 | Russell C. Jackson (Rusty) |
New script to create a filtered checkpoint to use to rebuild an edge server. This is not required unless you are filtering data via the server spec. If you are not doing that, then you can reseed the edge using the latest checkpoint of the master. |