#!/usr/local/bin/python # # This script is intended to give some degree of version control and user # permissions on jobs. # It is currently under development and is neither complete nor of "production" # quality. Please bear that in mind when using it. I would STRONGLY advise # running it against a test server initially, just to check that it will do # what you expect it to before turning it loose on your live system since # some of the operation it performs can be destructive, deleting Perforce # jobs for example. # # The script is intended to be run either from a scheduling app, like cron, # or by setting the "repeat" variable to "1" and the "runinterval" variable # to the number of seconds between runs. # # In order to use this script you must enable the "p4 logger". # To do this simply run: # # p4 counter logger 0 # # from a Perforce command line client. # # Due to the use this script makes of the logger functionality, running it in # conjunction with P4DTI and possibly any other defect tracker will cause both # to misbehave and miss data. SO DON'T DO IT!! # # Since this script will create a file for each job with the same name as the # job you should be careful to check that all of the job names you have are # valid file names on your OS. If you are using the default job naming scheme # of job00001, job000002, job000003, etc. then this will be fine, if however # you give your jobs other, possibly more descriptive, names (eg. Semaphore?) # then this is something you should be aware of. # If you do need to rename jobs for this reason then please do all of this # BEFORE enabling the Perforce logger mechanism, otherwise the script will try # to version jobs which no longer exist. # # Please bear in mind that the one operation this script can have no control # over is unauthorised users deleting jobs from Perforce. This is because by # the time the daemon comes to check the job, there is no job to check!! # In this case the daemon will log this fact to the command line and the log # file and mark the head rev of the versioned job file as deleted. This way # you could manually reinstate the Perforce job from the versioned file if # you wanted to. # # # A few configuration tips: # # 1) The "localpath" variable needs the double slashes (\\) on Windows machines # otherwise Python interprets them differently. You do NOT need double # forward slashes (//) in a Unix environment. # This variable should also match the "root" value of the client used to run # the script. # NOTE: you will have to create this directory manually as the script won't # do it for you. Perhaps this is something which can bee added to the # script in a future version. # # 2) The "client" variable should be set to the name of a precreated client on # the machine which will run this script. # # 3) The "user" variable should be a user with write permission on the area of # the depot where the versioned job files will be stored, ie. the "depotpath". # # 4) The "authorised" variable contains a list of the names of users who are # authorised to alter jobs. To have multiple users authorised to alter jobs # then include them all in the "authorised" variable, like so: # 'authorised': ['steve','admin','perforce'] # # 5) The "userfield" variable should be the name of a field in your jobspec # which is always set to the name of the last user to edit the job. # e.g. 109 Altered word 32 always # and should have a "Presets:" value of $user, e.g. Altered $user # THIS PART OF THE CONFIGURATION IS VITAL. WITHOUT THIS SUPPORT IN THE JOBSPEC # THE AUTHORISATION OF JOBS SIMPLY WON'T WORK AND THE SCRIPT WILL FAIL. # # # Config Variables - What they do. # user Perforce user to run commands as # client Perforce client to sync files to # localpath Local client files directory (same directory as client root) # jvpath Depot path where versioned job files are stored # authorised List of users authorised to make changes to jobs # userfield The field to check for who last edited the job # repeat Loop the script automatically # runinterval If so how long in seconds between runs # p4 The path to your p4 command # changedesc The text of the description field in submitted changelists # log A file to log messages in. Leave blank if you want # them all sent to stdout. # add A job added by an unauthorised user should be # added (add=1) or deleted (add=0). # revert A job altered by an unauthorised user should be # reverted (revert=1) in the Perforce job or ignored (revert=0). # I the changes are ignored in the Perforce job then the versioned # file which contains the job details will NOT be updated to reflect # the new contents of the job. This will only happen when the job is # altered by an authorised user. # alwaysprint Always write all messages to the console if this is set to 1, even # if they are also logged to a text file. If this is set to 0 and the # 'log' variable is set to a file then all messages will ONLY be logged # in the log file, not to STDOUT. # # Alter the values below to change the configuration # settings for you installation of this script. # config = {'user': 'steve', 'client': 'review-daemon', 'localpath': 'c:\\localwork\\vjobs\\', 'jvpath': '//depot/vjobs/', 'authorised': ['steve'], 'userfield': 'Altered', 'repeat': 1, 'runinterval': 30, 'p4': 'p4', 'changedesc': 'Jobs altered/created by Job Versioning Daemon', 'log': 'c:\\localwork\\vjobs\\vjobs.log', 'add': 1, 'revert': 0, 'alwaysprint': 1} # # End of configuration settings. # Don't alter anything below this line unless you are # altering some element of the scripts functionality, for # which you may need a reasonable understanding of Python. # # Import functionality from standard Python external modules from marshal import load from os import popen from re import match from string import count from time import ctime, time, sleep # Start class definitions class P4Command: def __init__(self): '''The class constructor''' # Build the config variable elements into # the start of the Perforce command line. self.command = config['p4'] + \ ' -u ' + config['user'] + \ ' -c ' + config['client'] + ' ' def P4Run(self, args, mode = 'r'): '''Run the command passed in the 'args' parameter. The mode of the command can be set if required, like in the RevertJob() function, but defaults to read.''' # Run the command we have been passed and return # a handle to the pipe that the results will be available on. return popen(self.command + args, mode) def P4Marshal(self, args): '''Return the dictionary from the marshalled output of the command passed in the 'args' parameter''' # Add the '-G' flag to the command passed to the funtion to output # Python marshalled object from the Perforce command. Return the # dictionary created by 'loading' this data. # NOTE: this only works for commands which will return a single object. return load(popen(self.command + ' -G ' + args, 'rb')) class P4Job(P4Command): def __init__(self, jobname): '''Class constructor''' # Call the base classes constructor. P4Command.__init__(self) # Initialise some instance variables. self.name = '' self.dict = {} self.have = '' self.action = '' self.name = jobname # Check if we already have a file for this job. self.have = self.P4Run(' have ' + config['localpath'] + self.name).readline() # Load the details of this job into a dictionary. self.dict = self.P4Marshal('job -o ' + self.name) # The message handling mechanism is currently pretty crude and simply # prints out what ever text it is sent to stdout and a log file if one # has been specified in the config variables. # The next step is to alter this to an email mechanism which will mail # the interested parties for each event of note and log them as well. def HandleMessage(self, msg): '''A farily simple message handler.Pass it a message string and it will print it out to console and/or log it in a file.''' # If a log file has been specified in the config varables... if config['log']: # open the log file for append... fh = open(config['log'], 'a') # write the current time and the message to the open file... fh.write(ctime(time()) + ' - ' + msg + '\n') # and close the log file. fh.close() # If the 'alwaysprint' config variable is set to 0 then... if not config['alwaysprint']: # drop out of this function at this point. return # if no log file is specified or if the 'alwaysprint' variable # is not set to 0, then print the current time and the message # to stdout. print ctime(time()) + ' - ' + msg + '\n' def GetAction(self): '''Determine what action has happened to this job based on whether we have a file for it and the contents of the dictionary, if there is one.''' # If we have neither a file for the job nor a Perforce record for the job... if not self.have and not self.dict: # Then this is some sort of error. self.action = 'error' return # If we do have a file for the job, but we don't have a Peforce record, # or we have an error in the dictionary for that record... if self.have and ((self.dict['code'] == 'error') or not self.dict): # Then this job was deleted in Perforce. self.action = 'delete' return # If we don't have a file, but we do have a Perforce job record... if not self.have and self.dict: # Then this job has been added. self.action = 'add' # We don't need the 'code' field for an add # so delete it from the dictionary. del self.dict['code'] return # Anything else must be an edit. self.action = 'edit' # We don't need the 'code' field for an edit either # so delete it from the dictionary. del self.dict['code'] def TakeAction(self): '''asdf''' # If the action is an error... if self.action == 'error': # just log a message. self.HandleMessage('Error with job ' + self.name + '.\n' + \ 'The job was probably created and deleted between runs.') return # If the action was a delete... if self.action == 'delete': # open the job file for delete and log a message. self.DeleteJobFile('Delete job file ' + self.name) return # Check whether the user making the changes is authorised to do so. if self.IsAuth(): # If they are and the action on the job was an add... if self.action == 'add': # Add the job file and log a message. self.AddJobFile('Add job file ' + self.name) return # Otherwise it must have been an edit, # so edit the job file and log a message. self.EditJob('Edit job file ' + self.name) return # If they aren't authorised and the action is an add... if self.action == 'add': # If the config variable 'add' is anything other than 0.. if config['add']: # Add the job file anyway and log the fact that # the job was created by an unauthorised user. self.AddJobFile('Add job file ' + self.name + \ '.\nThis job was created by an unauthorised user!') return # Otherwise, delete the job from Perforce and log a message. self.DeleteJob('Job ' + self.name + ' was created by an ' + \ 'unauthorised user.\n Deleting job from Perforce.') return # If we get to this point then the action must be an unauthorised edit, so # if the 'revert' config variable is set to anything other than 0... if config['revert']: # Revert the changes in the Perforce job self.RevertJob('Job ' + self.name + ' altered by an ' + \ 'unauthorised user. Changes reverted in Perforce job.') return # Otherwise, just log a message saying that the job was edited # by an unauthorised user, but don't write the changes to the job file. self.HandleMessage('Job ' + self.name + ' was altered by an ' + \ 'unauthorised user.\n' + \ 'The changes won\'t be written to the job file.') def IsAuth(self): '''Check if the user who altered the job has permission to do so''' # If the user who altered this job appears in the list # of authorised user set in the config variables... if self.dict[config['userfield']] in config['authorised']: return 1 # If not... return 0 def EditJob(self, msg = None): '''Open the job file for edit and write the changes out to it.''' # If any message was passed down from the caller... if msg: # call the HandleMessage() function and pass the message text on. self.HandleMessage(msg) # Open the file relating to this job for # edit in the Perforce default changelist. self.P4Run('edit ' + config['jvpath'] + self.name) # Call this function to write the altered job details to the jobs file. self.WriteJobFile() def DeleteJob(self, msg = None): '''Delete this job from Perforce''' # If any message was passed down from the caller... if msg: # Add a couple of trailing line feeds to the message... msg = msg + '\n\n' # then for each field in the job dictionary... for key in self.dict.keys: # Append the key and it's value to the message string. msg = msg + key + ': ' + self.dict[key] + '\n' # call the HandleMessage() function and # pass the altered message text on. self.HandleMessage(msg) # Delete the Perforce job. self.P4Run('job -d ' + self.name) def RevertJob(self, msg = None): '''Revert the Perforce job to it's state before the unauthorised user made their alterations.''' # If any message was passed down from the caller... if msg: # call the HandleMessage() function and pass the message text on. self.HandleMessage(msg) # Open a file handle to the Perforce job record... fh = self.P4Run('job -i -f', 'w') # For each line in the job file... for line in open(config['localpath'] + self.name, 'r').readlines(): # if the line doesn't contain a tab then it # needs one at the begining of the line... if '\t' not in line: # so prepend one... line = '\t' + line # and write the line to the opened file handle. fh.write(line) # Now close the file handle to commit the # changes to the Perforce job record. fh.close() def DeleteJobFile(self, msg = None): '''Delete a job file for a deleted job by opening it for delete in Perforce.''' # If any message was passed down from the caller... if msg: # call the HandleMessage() function and pass the message text on. self.HandleMessage(msg) # Open the file relating to the job for delete # in the Perforce default changelist. self.P4Run('delete ' + config['localpath'] + self.name) def WriteJobFile(self): '''Write the current job details to the workspace file.''' # Open a file for write, which will either create a new file # if one doesn't exist or overwrite an existing one. The file # will have the same name as the job name in Perforce. fh = open(config['localpath'] + self.name, 'w') # For each element of the job in the dictionary.. for key in self.dict.keys(): # Write the current key and it's value out to the job file. fh.write(key + ':\t' + str(self.dict[key]) + '\n') # Close the file. fh.close() def AddJobFile(self, msg = None): '''Ask for a new file for a newly created job and open it for add in Perforce''' # If any message was passed down from the caller... if msg: # call the HandleMessage() function and pass the message text on. self.HandleMessage(msg) # Call the WriteJobFile() function to create the new job file self.WriteJobFile() # Add the new job file to the Perforce default changelist. self.P4Run('add ' + config['jvpath'] + self.name) class P4Logger(P4Command): def __init__(self): '''The class constructor''' # Call the base classes constructor P4Command.__init__(self) # Initialise a list variable self.joblist = [] self.counter = '' # For every line in the output from the "p4 logger" command... for line in self.P4Run('logger').readlines(): # If we haven't already set the value of the counter instance variable... If not self.counter: # Grab the value of the logger counter at this point since it # is possible that more events will be generated while the rest # of the script is running and therefore leaving this until the # CleanUp() function, where it is used, may mean that we delete # events that we haven't checked on. self.counter = self.P4Run('counter logger').readline() # Find the events type and name. (type, job) = match(r'^\d+ (\S+) (.+)', line).groups() # If the type is a change, or we have already got a # note of this job name in the list variable... if (type == 'change') or (job in self.joblist): # then ignore the rest of this iteration of the loop continue # otherwise add this jobs name to the joblist. self.joblist.append(job) def CleanUp(self): '''Perform some last minute updates to the Perforce logger so that we don't see the exact same events next time we run.''' # Reset the logger to show only events after # the last one we checked on in this run of the script. # We're also stripping the lf off the end of the counter variable. self.P4Run('logger -c ' + self.counter[:-1] + ' -t logger') class P4JobVersion(P4Command): def __init__(self): '''The class constructor''' # We actually just want to call the base classes constructor P4Command.__init__(self) def InitialAdd(self): '''Check whether we currently have any versioned files, if not then add all of the current Perforce jobs as new versioned files and submit them''' if self.P4Run('have ' + config['jvpath'] + '...').readline(): # Tell the caller that we do indeed already have versioned job files. return 0 # For each job in Perforce for line in self.P4Run('jobs').readlines(): # Find the job name jobname = match(r'(\S+) on', line).group(1) # Create a new P4Job object for this job thisjob = P4Job(jobname) # Create a file for this job and add it to Perforce thisjob.AddJobFile() # Submit the added files self.SubmitFiles() # Tell the caller that we have added and # submitted all the existing Perforce jobs. return 1 def SubmitFiles(self): '''Submit the default changelist with a standard change description''' # Initialise a list to hold the change form newchange = [] # for each line in the change form... for changeline in self.P4Run('change -o').readlines(): # check for the standard change description in this line if count(changeline, "<enter description here>") > 0: # if its in this line then change this line to # the default job versioner change description. changeline = '\t%s\n' % config['changedesc'] # Add this line to the list variable which contains the new change form newchange.append(changeline) # Run the submit and pass in the chage form from the # list variable instead of invoking the users editor self.P4Run('submit -i', 'w').writelines(newchange) def MainLoop(self): '''This is the main loop which handles the version of the job files''' # Create a new P4Logger object logger = P4Logger() # For each job in the logger objects joblist for job in logger.joblist: # Create a new P4Job object for this job, passing in the jobs name. thisjob = P4Job(job) # Work out what action has occured to this job in Perforce thisjob.GetAction() # Perform the neccesary steps to complete the action taken # with this job in Perforce thisjob.TakeAction() # We have finished with this particular job, so delete the object del thisjob # Once we have taken the relevent action for all the jobs in the # loggers joblist we submit any changes pending in the default changelist self.SubmitFiles() # Run the clear up routine for the logger object logger.CleanUp() # Delete the logger object del logger ########## ## Main ## ########## #if the repeat variable is set we loop forever while config['repeat']: # create a new P4JobVersion object versioner = P4JobVersion() # if the script has never been run before then add all # the files for the existing jobs, otherwise... if not versioner.InitialAdd(): # Run the main versioning loop versioner.MainLoop() # we've finished with this versioner, so delete it. del versioner # Inform the user what is going on print 'Sleeping for %d seconds' % config['runinterval'] # wait for the specified number of seconds before running this loop again sleep(config['runinterval']) # if the repeat variable is not set then only run once else: # create a new P4JobVersion object versioner = P4JobVersion() # if the script has never been run before then add all # the files for the existing jobs, otherwise... if not versioner.InitialAdd(): # Run the main versioning loop versioner.MainLoop()
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#10 | 1322 | Steve Borrett | Only submit if we need to, so check "p4 opened" before trying the submit. | ||
#9 | 1321 | Steve Borrett |
Altered the comments, since they were a bit excessive. If you really want to see a version with every line of code fully commented then you should look at the previous revision. |
||
#8 | 1269 | Steve Borrett |
Fixed a minor bug in the P4Logger class. There was an "if" with a capital "I". |
||
#7 | 1252 | Steve Borrett |
Moved the logger counter checking to the constructor of the P4Logger class so that there is less chance of more events occuring before the counters value is recorded and the logger reset in the CleanUp() method. Added the config variable 'alwaysprint'. This will mean that all messages are always written to STDOUT, even if a log file is specified. We now write the entire dictionary to the message in the DeleteJob() method of the P4Job class instead of ignoring the 'code' field. Because, why not... Finished commenting the code and config variables and did some minor rearranging of the code layout to make things more obvious and readable. |
||
#6 | 1202 | Steve Borrett |
Moved the code which decides what action to take with a job to a TakeAction() method of the P4Job class. Added a call to the above function to the MainLoop() function of the of the P4Versioner class Commented the code in the main program body and the P4Versioner class and it's methods. |
||
#5 | 1201 | Steve Borrett |
Added a "revert" config variable which allows users to either ignore alterations to jobs by unauthorised users, or revert thier changes to the previous state. Reformatted the code to fit in an 80 column screen. Altered the P4Job.DeleteJob() function to include the job details in the logged message before the job is deleted from Perforce. Added + '\n' to the end of the print statement in P4Job.HandleMessage() to ensure that logged messages always end with a LF. Made some changes and additions to the comments at the start of the script. |
||
#4 | 1082 | Steve Borrett |
Alter imports to only import required functions. Alter code to reflect above import changes. Add config variable "add" to determine the action to be taken when an unauthorised user creates a job. Altered code to reflect the above. |
||
#3 | 1077 | Steve Borrett |
Reorganisation of code to better reflect class hierarchy. No functional alterations. |
||
#2 | 1076 | Steve Borrett |
An update to ensure all the latest fixes are in the public depot version of the script, since I am mostly controling this on my local Perforce server. |
||
#1 | 1074 | Steve Borrett |
Delete original job versioning script and add in the new improved version 2 script. This script has vast differences from the original and is now totally class based. As far as I can tell all of the issues in the original script have been resolved in this incarnation, including such things as being able to delete jobs from Perforce. The job file will simply be "p4 delete"ed and the job itself will be deleted from Perforce. This means that although there is still no protections possible on being able to delete jobs, there is a record of what they were prior to their deletion. One thing to be wary of with this version of the script is that jobs created by an unauthorised user are currently simply removed from Perforce and no trace of them exists. This is possibly a little extreme but for now is the easist solution to implement. See the to do list at the bottom of the script for things which I am currently working on for this script. |