#!/usr/local/bin/python # # This script is intended to give some degree of version control and user # permissions on jobs. # It is currently under development and is neither complete nor of "production" # quality. Please bear that in mind when using it. I would STRONGLY advise # running it against a test server initially, just to check that it will do # what you expect it to before turning it loose on your live system since # some of the operation it performs can be destructive, deleting Perforce # jobs for example. # # The script is intended to be run either from a scheduling app, like cron, # or by setting the "repeat" variable to "1" and the "runinterval" variable # to the number of seconds between runs. # # In order to use this script you must enable the "p4 logger". # To do this simply run: # # p4 counter logger 0 # # from a Perforce command line client. # # Due to the use this script makes of the logger functionality, running it in # conjunction with P4DTI and possibly any other defect tracker will cause both # to misbehave and miss data. SO DON'T DO IT!! # # Since this script will create a file for each job with the same name as the # job you should be careful to check that all of the job names you have are # valid file names on your OS. If you are using the default job naming scheme # of job00001, job000002, job000003, etc. then this will be fine, if however # you give your jobs other, possibly more descriptive, names (eg. Semaphore?) # then this is something you should be aware of. # If you do need to rename jobs for this reason then please do all of this # BEFORE enabling the Perforce logger mechanism, otherwise the script will try # to version jobs which no longer exist. # # Please bear in mind that the one operation this script can have no control # over is unauthorised users deleting jobs from Perforce. This is because by # the time the daemon comes to check the job, there is no job to check!! # In this case the daemon will log this fact to the command line and the log # file and mark the head rev of the versioned job file as deleted. This way # you could manually reinstate the Perforce job from the versioned file if # you wanted to. # # # A few configuration tips: # # 1) The "localpath" variable needs the double slashes (\\) on Windows machines # otherwise Python interprets them differently. You do NOT need double # forward slashes (//) in a Unix environment. # This variable should also match the "root" value of the client used to run # the script. # NOTE: you will have to create this directory manually as the script won't # do it for you. Perhaps this is something which can bee added to the # script in a future version. # # 2) The "client" variable should be set to the name of a precreated client on # the machine which will run this script. # # 3) The "user" variable should be a user with write permission on the area of # the depot where the versioned job files will be stored, ie. the "depotpath". # # 4) The "authorised" variable contains a list of the names of users who are # authorised to alter jobs. To have multiple users authorised to alter jobs # then include them all in the "authorised" variable, like so: # 'authorised': ['steve','admin','perforce'] # # 5) The "userfield" variable should be the name of a field in your jobspec # which is always set to the name of the last user to edit the job. # e.g. 109 Altered word 32 always # and should have a "Presets:" value of $user, e.g. Altered $user # THIS PART OF THE CONFIGURATION IS VITAL. WITHOUT THIS SUPPORT IN THE JOBSPEC # THE AUTHORISATION OF JOBS SIMPLY WON'T WORK AND THE SCRIPT WILL FAIL. # # # Config Variables - What they do. # user Perforce user to run commands as # client Perforce client to sync files to # localpath Local client files directory (same directory as client root) # jvpath Depot path where versioned job files are stored # authorised List of users authorised to make changes to jobs # userfield The field to check for who last edited the job # repeat Loop the script automatically # runinterval If so how long in seconds between runs # p4 The path to your p4 command # changedesc The text of the description field in submitted changelists # log A file to log messages in. Leave blank if you want # them all sent to stdout. # add A job added by an unauthorised user should be # added (add=1) or deleted (add=0). # revert A job altered by an unauthorised user should be # reverted (revert=1) in the Perforce job or ignored (revert=0). # I the changes are ignored in the Perforce job then the versioned # file which contains the job details will NOT be updated to reflect # the new contents of the job. This will only happen when the job is # altered by an authorised user. # alwaysprint Always write all messages to the console if this is set to 1, even # if they are also logged to a text file. If this is set to 0 and the # 'log' variable is set to a file then all messages will ONLY be logged # in the log file, not to STDOUT. # # Alter the values below to change the configuration # settings for you installation of this script. # config = {'user': 'steve', 'client': 'review-daemon', 'localpath': 'c:\\localwork\\vjobs\\', 'jvpath': '//depot/vjobs/', 'authorised': ['steve'], 'userfield': 'Altered', 'repeat': 1, 'runinterval': 20, 'p4': 'p4', 'changedesc': 'Jobs altered/created by Job Versioning Daemon', 'log': 'c:\\localwork\\vjobs\\vjobs.log', 'add': 1, 'revert': 0, 'alwaysprint': 1} # # End of configuration settings. # Don't alter anything below this line unless you are # altering some element of the scripts functionality, for # which you may need a reasonable understanding of Python. # # Import functionality from standard Python external modules from marshal import load from os import popen from re import match from string import count from time import ctime, time, sleep # Start class definitions class P4Command: def __init__(self): '''The class constructor''' # Build the config variable elements into # the start of the Perforce command line. self.command = config['p4'] + \ ' -u ' + config['user'] + \ ' -c ' + config['client'] + ' ' def P4Run(self, args, mode = 'r'): '''Run a Perforce command.''' # Run the command we have been passed and return # a handle to the pipe that the results will be available on. return popen(self.command + args, mode) def P4Marshal(self, args): '''Run a Perforce command and return a Python marshalled dictionary of the results.''' # Add the '-G' flag to the command passed to the funtion to output # Python marshalled object from the Perforce command. Return the # dictionary created by 'loading' this data. # NOTE: this only works for commands which will return a single object. return load(popen(self.command + ' -G ' + args, 'rb')) class P4Job(P4Command): def __init__(self, jobname): '''Class constructor''' # Call the base classes constructor. P4Command.__init__(self) # Initialise some instance variables. self.name = '' self.dict = {} self.have = '' self.action = '' self.name = jobname # Check if we already have a file for this job. self.have = self.P4Run(' have ' + config['localpath'] + self.name).readline() # Load the details of this job into a dictionary. self.dict = self.P4Marshal('job -o ' + self.name) # The message handling mechanism is currently pretty crude and simply # prints out what ever text it is sent to stdout and a log file if one # has been specified in the config variables. # The next step is to alter this to an email mechanism which will mail # the interested parties for each event of note and log them as well. def HandleMessage(self, msg): '''A farily simple message handler.Pass it a message string and it will print it out to console and/or log it in a file.''' # If a log file has been specified in the config varables... if config['log']: fh = open(config['log'], 'a') # write the current time and the message to the open file... fh.write(ctime(time()) + ' - ' + msg + '\n') fh.close() # If the 'alwaysprint' config variable is set to 0 then... if not config['alwaysprint']: return # if no log file is specified or if the 'alwaysprint' variable # is not set to 0, then print the current time and the message # to stdout. print ctime(time()) + ' - ' + msg + '\n' def GetAction(self): '''Determine what action has happened to this job.''' # If we have neither a file for the job nor a Perforce record for the job... if not self.have and not self.dict: self.action = 'error' return # If we do have a file for the job, but we don't have a Peforce record, # or we have an error in the dictionary for that record... if self.have and ((self.dict['code'] == 'error') or not self.dict): self.action = 'delete' return # If we don't have a file, but we do have a Perforce job record... if not self.have and self.dict: self.action = 'add' # We don't need the 'code' field for an add # so delete it from the dictionary. del self.dict['code'] return # Anything else must be an edit. self.action = 'edit' del self.dict['code'] def TakeAction(self): '''Act on the job based on the action which occured.''' if self.action == 'error': # just log a message. self.HandleMessage('Error with job ' + self.name + '.\n' + \ 'The job was probably created and deleted between runs.') return if self.action == 'delete': # open the job file for delete and log a message. self.DeleteJobFile('Delete job file ' + self.name) return # Check whether the user making the changes is authorised to do so. if self.IsAuth(): if self.action == 'add': # Add the job file and log a message. self.AddJobFile('Add job file ' + self.name) return # Otherwise it must have been an edit, # so edit the job file and log a message. self.EditJob('Edit job file ' + self.name) return # If they aren't authorised if self.action == 'add': # If the config variable 'add' is anything other than 0.. if config['add']: # Add the job file anyway and log the fact that # the job was created by an unauthorised user. self.AddJobFile('Add job file ' + self.name + \ '.\nThis job was created by an unauthorised user!') return # Otherwise, delete the job from Perforce and log a message. self.DeleteJob('Job ' + self.name + ' was created by an ' + \ 'unauthorised user.\n Deleting job from Perforce.') return # If we get to this point then the action must be an unauthorised edit, so # if the 'revert' config variable is set to anything other than 0... if config['revert']: # Revert the changes in the Perforce job self.RevertJob('Job ' + self.name + ' altered by an ' + \ 'unauthorised user. Changes reverted in Perforce job.') return # Otherwise, just log a message saying that the job was edited # by an unauthorised user, but don't write the changes to the job file. self.HandleMessage('Job ' + self.name + ' was altered by an ' + \ 'unauthorised user.\n' + \ 'The changes won\'t be written to the job file.') def IsAuth(self): '''Check if the user who altered the job has permission to do so''' if self.dict[config['userfield']] in config['authorised']: return 1 return 0 def EditJob(self, msg = None): '''Open the job file for edit and write the changes out to it.''' # If any message was passed down from the caller... if msg: self.HandleMessage(msg) # Open the file relating to this job for edit. self.P4Run('edit ' + config['jvpath'] + self.name) # Write the altered job details to the file. self.WriteJobFile() def DeleteJob(self, msg = None): '''Delete this job from Perforce''' if msg: # Add a couple of trailing line feeds to the message... msg = msg + '\n\n' # then for each field in the job dictionary... for key in self.dict.keys: # Append the key and it's value to the message string. msg = msg + key + ': ' + self.dict[key] + '\n' self.HandleMessage(msg) # Delete the Perforce job. self.P4Run('job -d ' + self.name) def RevertJob(self, msg = None): '''Revert the Perforce job to the state contained in the versioned file.''' if msg: self.HandleMessage(msg) # Open a file handle to the Perforce job record... fh = self.P4Run('job -i -f', 'w') for line in open(config['localpath'] + self.name, 'r').readlines(): # if the line doesn't contain a tab then it # needs one at the begining of the line... if '\t' not in line: line = '\t' + line # write the line to the opened file handle. fh.write(line) fh.close() def DeleteJobFile(self, msg = None): '''Delete a job file.''' # If any message was passed down from the caller... if msg: # call the HandleMessage() function and pass the message text on. self.HandleMessage(msg) # Open the file relating to the job for delete # in the Perforce default changelist. self.P4Run('delete ' + config['localpath'] + self.name) def WriteJobFile(self): '''Write the current job details to the workspace file.''' fh = open(config['localpath'] + self.name, 'w') for key in self.dict.keys(): # Write the current key and it's value out to the job file. fh.write(key + ':\t' + str(self.dict[key]) + '\n') fh.close() def AddJobFile(self, msg = None): '''Add a new file for a new Perforce job.''' if msg: self.HandleMessage(msg) self.WriteJobFile() # Add the new job file to the Perforce default changelist. self.P4Run('add ' + config['jvpath'] + self.name) class P4Logger(P4Command): def __init__(self): '''The class constructor''' P4Command.__init__(self) self.joblist = [] self.counter = '' # For every line in the output from the "p4 logger" command... for line in self.P4Run('logger').readlines(): # Find the events type and name. (type, job) = match(r'^\d+ (\S+) (.+)', line).groups() # If the type is a change, or we have already got a # note of this job name in the list variable... if (type == 'change') or (job in self.joblist): continue # otherwise add this jobs name to the joblist. self.joblist.append(job) def CleanUp(self): '''Perform some last minute clean up.''' # Reset the logger to show only events after # the last one we checked on in this run of the script. self.counter = self.P4Run('counter logger').readline() self.P4Run('logger -c ' + self.counter[:-1] + ' -t logger', 'w') class P4JobVersion(P4Command): def __init__(self): '''The class constructor''' P4Command.__init__(self) def InitialAdd(self): '''Check whether we currently have any versioned files, if not then add all of the current Perforce jobs as new versioned files and submit them''' if self.P4Run('have ' + config['jvpath'] + '...').readline(): # If we currently have versioned job files. return 0 for line in self.P4Run('jobs').readlines(): # Find the job name jobname = match(r'(\S+) on', line).group(1) # Create a new P4Job object for this job thisjob = P4Job(jobname) # Create a file for this job and add it to Perforce thisjob.AddJobFile() # Submit the added files self.SubmitFiles() return 1 def SubmitFiles(self): '''Submit the default changelist with a standard change description''' newchange = [] for changeline in self.P4Run('change -o').readlines(): # check for the standard change description in this line if count(changeline, "<enter description here>") > 0: # Change the description to the default supplied in the config. changeline = '\t%s\n' % config['changedesc'] # Add this line to the list variable which contains the new change form newchange.append(changeline) # Run the submit and pass in the chage form from the # list variable instead of invoking the users editor self.P4Run('submit -i', 'w').writelines(newchange) def MainLoop(self): # Create a new P4Logger object logger = P4Logger() # For each job in the logger objects joblist for job in logger.joblist: # Create a new P4Job object for this job thisjob = P4Job(job) thisjob.GetAction() thisjob.TakeAction() del thisjob if self.P4Run('opened').readline(): self.SubmitFiles() # only submit if we need to. logger.CleanUp() # only clean up if we need to. # Delete the logger object del logger ########## ## Main ## ########## #if the repeat variable is set we loop forever while config['repeat']: versioner = P4JobVersion() if not versioner.InitialAdd(): versioner.MainLoop() del versioner print 'Sleeping for %d seconds' % config['runinterval'] # wait for the specified number of seconds before running this loop again sleep(config['runinterval']) # if the repeat variable is not set then only run once else: versioner = P4JobVersion() if not versioner.InitialAdd(): versioner.MainLoop()
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#10 | 1322 | Steve Borrett | Only submit if we need to, so check "p4 opened" before trying the submit. | ||
#9 | 1321 | Steve Borrett |
Altered the comments, since they were a bit excessive. If you really want to see a version with every line of code fully commented then you should look at the previous revision. |
||
#8 | 1269 | Steve Borrett |
Fixed a minor bug in the P4Logger class. There was an "if" with a capital "I". |
||
#7 | 1252 | Steve Borrett |
Moved the logger counter checking to the constructor of the P4Logger class so that there is less chance of more events occuring before the counters value is recorded and the logger reset in the CleanUp() method. Added the config variable 'alwaysprint'. This will mean that all messages are always written to STDOUT, even if a log file is specified. We now write the entire dictionary to the message in the DeleteJob() method of the P4Job class instead of ignoring the 'code' field. Because, why not... Finished commenting the code and config variables and did some minor rearranging of the code layout to make things more obvious and readable. |
||
#6 | 1202 | Steve Borrett |
Moved the code which decides what action to take with a job to a TakeAction() method of the P4Job class. Added a call to the above function to the MainLoop() function of the of the P4Versioner class Commented the code in the main program body and the P4Versioner class and it's methods. |
||
#5 | 1201 | Steve Borrett |
Added a "revert" config variable which allows users to either ignore alterations to jobs by unauthorised users, or revert thier changes to the previous state. Reformatted the code to fit in an 80 column screen. Altered the P4Job.DeleteJob() function to include the job details in the logged message before the job is deleted from Perforce. Added + '\n' to the end of the print statement in P4Job.HandleMessage() to ensure that logged messages always end with a LF. Made some changes and additions to the comments at the start of the script. |
||
#4 | 1082 | Steve Borrett |
Alter imports to only import required functions. Alter code to reflect above import changes. Add config variable "add" to determine the action to be taken when an unauthorised user creates a job. Altered code to reflect the above. |
||
#3 | 1077 | Steve Borrett |
Reorganisation of code to better reflect class hierarchy. No functional alterations. |
||
#2 | 1076 | Steve Borrett |
An update to ensure all the latest fixes are in the public depot version of the script, since I am mostly controling this on my local Perforce server. |
||
#1 | 1074 | Steve Borrett |
Delete original job versioning script and add in the new improved version 2 script. This script has vast differences from the original and is now totally class based. As far as I can tell all of the issues in the original script have been resolved in this incarnation, including such things as being able to delete jobs from Perforce. The job file will simply be "p4 delete"ed and the job itself will be deleted from Perforce. This means that although there is still no protections possible on being able to delete jobs, there is a record of what they were prior to their deletion. One thing to be wary of with this version of the script is that jobs created by an unauthorised user are currently simply removed from Perforce and no trace of them exists. This is possibly a little extreme but for now is the easist solution to implement. See the to do list at the bottom of the script for things which I am currently working on for this script. |