#!/usr/bin/env python
"""
This script will unload clients that have not been accessed since the number
of weeks specified in the maintenance.cfg file.
This version of unload clients will delete clients with exclusive checkouts
since unload will not unload those clients.
It also deletes shelves from the clients to be deleted since delete will not
delete a client with a shelf.
"""
import os
import re
import sys
import string
import accessdates
import platform
import sdputils
if len(sys.argv) > 1:
SDP_INSTANCE = str(sys.argv[1])
else:
SDP_INSTANCE = '1'
utils = sdputils.SDPUtils(SDP_INSTANCE)
p4 = utils.p4
utils.login()
cfgweeks = int(utils.get('weeks'))
###############################################################################
def unload_clients( command, del_command ):
clients = []
accessdates.createlist("clients", "client", cfgweeks)
# Generate list of exclusively opened files.
os.system('%s -ztag opened -a | grep -A 3 "+F*l" > excllist.txt' % (p4))
if os.path.isfile("excllist.txt"):
input = open( "excllist.txt", "r" )
# Create list of clients with exclusively opened files.
for line in input.readlines():
line = line.strip()
if re.search("\.\.\. client ", line):
client = line[11:]
client = client.strip()
clients.append( client )
input.close()
os.remove( "excllist.txt" )
# Read list of clients to be unloaded.
if os.path.isfile("clients.txt"):
clientsfile = open( "clients.txt", "r" )
for client in clientsfile.readlines():
client = client.strip()
# Avoid deleting or affecting Swarm clients.
if( "swarm-project" in client.lower() ):
continue
# Delete clients with exclusively opened files since unload won't unload them.
if client in clients:
client = re.sub("\$", "\\\$", client)
os.system('%s client -f -Fd %s' % (p4, client))
else:
client = re.sub("\$", "\\\$", client)
os.system(('%s unload -f -L -z -c %s') % (p4, client))
clientsfile.close()
os.remove( "clients.txt")
###############################################################################
# main
if __name__ == '__main__':
# Generate list of clients to be unloaded.
unload_clients( "clients", "client" )
| # | Change | User | Description | Committed | |
|---|---|---|---|---|---|
| #10 | 33526 | Russell C. Jackson (Rusty) |
Fix unconditional maintenance emails; add jsonl structured logging across Maintenance/ - Maintenance/maintenance and Maintenance/edge_maintenance (run daily via cron, despite the "weekly" naming) were emailing the full run log on every single run, success or not, because every step was guarded with `|| true` and never reported failure anywhere. Added a run_step helper that tracks real failures and wires them into sdp_set_status/ sdp_set_error; the end-of-run email now only fires when something actually failed (subject changed to "... FAILED" to make that obvious). - Every other script in Maintenance/ (39 Python scripts + create_p4_filelist.sh + email.sh; sdputils.py excluded as a pure library with no __main__) now emits the same jsonl structured-logging record as the cron scripts in common/bin, via sdp_structured_log.py's run() context manager (auto-emits success/failure on exit, sys.exit(N), or unhandled exception) or, for the two shell scripts, sdp_structured_log.sh's sdp_log_init (auto ERR/EXIT trap coverage). - remove_empty_pending_changes.py: the initial pending-changes listing failure path did a bare `return` (swallowed by the auto-emit machinery, since a normal return maps to exit code 0/success); now returns 1, and per-change delete failures inside the loop also flip the overall exit code, so a partially-failed run is correctly logged as a failure instead of a silent success. - pymail.py: usage() called a bare sys.exit() (exit code 0) for both the -h/help path AND real bad-invocation paths (missing required args, getopt errors) - meaning a bad invocation reported success both to the OS and to the structured log. usage() now takes an explicit code (default 0, preserving -h's exit-0 convention); the two real error call sites pass a nonzero code. |
||
| #9 | 32423 | Russell C. Jackson (Rusty) |
Modernize SDP maintenance scripts: security, correctness, and Python 3 - Replace all os.system() and os.popen() calls with subprocess.run() using argument lists to eliminate shell injection vulnerabilities - Fix critical bugs: broken indentation in convert_label_to_autoreload.py, malformed print() in p4lock/p4unlock, wrong variable in isitalabel, format string typo in maintain_user_from_groups - Add p4 property alias and shared SKIP_USERS constant to sdputils.py - Add try/finally with p4.disconnect() to all P4Python scripts - Replace bare except: with specific exception types throughout - Update all shebangs to python3, remove unnecessary __future__ imports - Use context managers for all file handle operations - Replace from subprocess import * with explicit imports |
||
| #8 | 32388 | Russell C. Jackson (Rusty) | Updates using Claude.ai to clean up the code, reduce duplication, enhanace security, and use current standards. | ||
| #7 | 31833 | Russell C. Jackson (Rusty) | A couple of additional fixes. | ||
| #6 | 31825 | Russell C. Jackson (Rusty) | Converted to use P4 api and to use only python 3. | ||
| #5 | 30843 | Russell C. Jackson (Rusty) | Updated to use parameters for the user and client commands. | ||
| #4 | 25066 | Russell C. Jackson (Rusty) | Added check for swarm clients to skip unloading them. | ||
| #3 | 25025 | Russell C. Jackson (Rusty) | Fixed typo in sdputils.py and forced cfgweeks to be an int. | ||
| #2 | 24675 | Russell C. Jackson (Rusty) |
Fixed bugs in sdputils.py and scripts using it. Converted to standard 2 space spacing, removed copyright stuff. |
||
| #1 | 22693 | Russell C. Jackson (Rusty) |
Branched a Unix only version of the SDP. Removed extra items to create a cleaner tree. Moved a few items around to make more sense without Windows in the mix. |
||
| //guest/perforce_software/sdp/dev/Maintenance/unload_clients_with_delete.py | |||||
| #5 | 22006 | C. Thomas Tyler |
Removed hard-coding assumption of an Instance named 1. Now, one of two things must be true: P4BIN must be defined, as it would be if the SDP envrionment is loaded in the usual way (source p4_vars N), or else the 'p4' executable must be found in the path. Avoided removal of Swarm workspaces. |
||
| #4 | 16029 | C. Thomas Tyler |
Routine merge to dev from main using: p4 merge -b perforce_software-sdp-dev |
||
| #3 | 13906 | C. Thomas Tyler |
Normalized P4INSTANCE to SDP_INSTANCE to get Unix/Windows implementations in sync. Reasons: 1. Things that interact with SDP in both Unix and Windows environments shoudn't have to account for this obscure SDP difference between Unix and Windows. (I came across this doing CBD work). 2. The Windows and Unix scripts have different variable names for defining the same concept, the SDP instance. Unix uses P4INSTANCE, while Windows uses SDP_INSTANCE. 3. This instance tag, a data set identifier, is an SDP concept. I prefer the SDP_INSTANCE name over P4INSTANCE, so I prpose to normalize to SDP_INSTANCE. 4. The P4INSTANCE name makes it look like a setting that might be recognized by the p4d itself, which it is not. (There are other such things such as P4SERVER that could perhaps be renamed as a separate task; but I'm not sure we want to totally disallow the P4 prefix for variable names. It looks too right to be wrong in same cases, like P4BIN and P4DBIN. That's a discussion for another day, outside the scope of this task). Meanwhile: * Fixed a bug in the Windows 2013.3 upgrade script that was referencing undefined P4INSTANCE, as the Windows environment defined only SDP_INSTANCE. * Had P4INSTANCE been removed completely, this change would likely cause trouble for users doing updates for existing SDP installations. So, though it involves slight technical debt, I opted to keep a redundant definition of P4INSTANCE in p4_vars.template, with comments indicating SDP_INSTANCE should be used in favor of P4INSTANCE, with a warning that P4INSTANCE may go away in a future release. This should avoid unnecessary upgrade pain. * In mkdirs.sh, the varialbe name was INSTANCE rather than SDP_INSTANCE. I changed that as well. That required manual change rather than sub/replace to avoid corrupting other similar varialbe names (e.g. MASTERINSTANCE). This is a trivial change technically (a substitute/replace, plus tweaks in p4_vars.template), but impacts many files. |
||
| #2 | 11477 | Russell C. Jackson (Rusty) |
Updated to use /usr/bin/env python Added workshop header. Changed cfg to config. |
||
| #1 | 11464 | Russell C. Jackson (Rusty) | Current maintenance scripts. | ||