A Journal Monitoring Daemon
Written for Python 2.3 on *NIX platforms
Version 1.1
Placed into the Public Domain
This tool monitors the live journal file from a Perforce server, and executes a chain of registered listeners based on certain events. It is capable of correctly handling a journal file being truncated by searching for previous journal files (either compressed or not).
The first step is to initialize the Settings.py file. You may either edit the Settings.py.template file by hand, or run the CreateSettings.py application to do it for you, based on your existing environment variables. This should be the only file you need to alter.
If you wish to use the hot-backup listener, then you should follow the steps below for setting up an SSH communication between the two servers. This will mean that both servers should use a minimal-access user for communication.
Since the tool will look in checkpointed journal files, it is a good idea to keep at least the most recent of those around, so that the daemon can go back and look up any data it may have missed. If something happens that causes the daemon to skip several journal files, including one that was deleted, then you will need to manually intervene to correct the situation.
Finally, you can extend the tool yourself with custom listeners. See that section below.
The daemon's main logic is handling the journal file, as it is updated and truncated during normal checkpoint operations. The daemon will persist a file on the machine running it that describes where it last left off (journal file sequence number and line number), so that the daemon can correctly recover from either a reboot, abnormal shutdown, or any other possible terminating event.
This little "where was I" file is very important. It's what ensures that the remote system is passed exactly the information it needs. If this file doesn't exist, then the tool will look backwards through the journals to find the one corresponding to the checkpoint number specified in Setup.py, under "SERVER.lastCheckpoint". This number should represent the last checkpoint that was recovered on the remote server.
If the tool fails to find a journal file in between the current journal and either the where-was-I file or the lastCheckpoint, then it will fail. You will need to manually recover from this point.
The daemon can quit on its own due to several reasons. The most common is a mis-configured Setup.py file.
Another issue, which can mean serious investigation, is when the daemon attempts to find a journal file that is no longer on the host computer. In the case of a simple hot-backup usage, you will need to restore the remote machine from a recent checkpoint, and update the Setup.py file to this checkpoint number. Other tools, such as e-mail reviewing, you may want to simply send a message to your users saying that no e-mails were sent during this time period, and that they'll have to perform a manual monitor themselves.
The tool parses through the journal files, invoking the registered list of listeners on each line encountered.
The list of registered listeners is stored in Setup.py, in the JOURNAL class. The "listeners" list contains strings of the names of listeners (including module name) to load.
Each listener is constructed once at daemon start-up time, and should extend the listener.Listener class. For each line of the Journal that is processed, first the tool invokes the listener's "isHandled(self, obj)" method (more on the "obj" argument later). If this method returns True, then the tool calls the "listen(self,obj)" method to handle the journal line.
If the tool notices a "quiet" period where no Journal activity occurs, the tool will call each listener's "noData(self, hadData)" method. Here, hadData is True if this is the first "noData" call after a period of activity.
The object passed to the "isHandled" and "listen" methods is a Journal.JournalEntry instance. This contains the following interesting attributes:
True if
there was a Journal rotation right before this entryTrue if we expect
more lines in this entryDefinitions:
$REMOTE = remote machine to log into
$RUSER = remote user on the remote machine to log in as.
From the machine that will run ssh:
# Create the keys for the local machine
$ ssh-keygen -t rsa
(use default location, and no passphrase)
# Install the ssh keys on the remote machine.
$ scp ~/.ssh/id_rsa.pub $RUSER@$REMOTE:./pubkey
(this will force you to login and authenticate the server,
which must be done at least once)
$ ssh $RUSER@$REMOTE "mkdir ~/.ssh && chmod 700 ~/.ssh ; cat ~/pubkey >> ~/.ssh/authorized_keys ; chmod 600 ~/.ssh/authorized_keys ; rm ~/pubkey"
(this will force you to login again)
$ ssh $RUSER@$REMOTE
(you shouldn't need to log in. If you do, check the local
and remote security files for instructions)
The authors are in no way liable for any damage this tool might do. You've been warned: use at your own risk. Standard administration procedures should require that you sufficiently test out any new tool before bringing it into your production environment. Remember to keep good backups. This tool probably won't eat your lunch, but you still need to be careful.