<html>
<head>
<title>logDaemon - A P4 Journal Monitoring Daemon</title>
</head>
<body>
<h1>logDaemon</h1>
<p>
A Journal Monitoring Daemon<br/>
Written for Python 2.3 on *NIX platforms</br>
Version 1.1</br>
Placed into the Public Domain
</p>
<h1>About</h1>
<p>
This tool monitors the live journal file from a Perforce server, and executes a
chain of registered listeners based on certain events. It is capable of
correctly handling a journal file being truncated by searching for previous
journal files (either compressed or not).
</p>
<h1>Using The Tool</h1>
<p>
The first step is to initialize the Settings.py file. You may either edit the
Settings.py.template file by hand, or run the CreateSettings.py application
to do it for you, based on your existing environment variables. This should
be the only file you need to alter.
</p>
<p>
If you wish to use the hot-backup listener, then you should follow the steps
below for setting up an SSH communication between the two servers. This
will mean that both servers should use a minimal-access user for
communication.
</p>
<p>
Since the tool will look in checkpointed journal files, it is a good idea to
keep at least the most recent of those around, so that the daemon can
go back and look up any data it may have missed. If something happens that
causes the daemon to skip several journal files, including one that was
deleted, then you will need to manually intervene to correct the situation.
</p>
<p>
Finally, you can extend the tool yourself with custom listeners. See that
section below.
</p>
<h1>Tool Details</h1>
<p>
The daemon's main logic is handling the journal file, as it is updated and
truncated during normal checkpoint operations. The daemon will persist a
file on the machine running it that describes where it last left off
(journal file sequence number and line number), so that the daemon can
correctly recover from either a reboot, abnormal shutdown, or any other
possible terminating event.
</p>
<p>
This little "where was I" file is very important. It's what ensures that the
remote system is passed exactly the information it needs. If this file doesn't
exist, then the tool will look backwards through the journals to find the
one corresponding to the checkpoint number specified in Setup.py, under
"SERVER.lastCheckpoint". This number should represent the last checkpoint
that was recovered on the remote server.
</p>
<p>
If the tool fails to find a journal file in between the current journal and
either the where-was-I file or the lastCheckpoint, then it will fail. You
will need to manually recover from this point.
</p>
<h1>Correcting From Errors</h1>
<p>
The daemon can quit on its own due to several reasons. The most common is
a mis-configured Setup.py file.
</p>
<p>
Another issue, which can mean serious investigation, is when the daemon
attempts to find a journal file that is no longer on the host computer.
In the case of a simple hot-backup usage, you will need to restore the
remote machine from a recent checkpoint, and update the Setup.py file
to this checkpoint number. Other tools, such as e-mail reviewing,
you may want to simply send a message to your users saying that no
e-mails were sent during this time period, and that they'll have to
perform a manual monitor themselves.
</p>
<h1>Adding Listeners</h1>
<p>
The tool parses through the journal files, invoking the registered list of
listeners on each line encountered.
</p>
<p>
The list of registered listeners is stored in Setup.py, in the JOURNAL
class. The "listeners" list contains strings of the names of listeners
(including module name) to load.
</p>
<p>
Each listener is constructed once at daemon start-up time, and should
extend the listener.Listener class. For each line of the Journal that is
processed, first the tool invokes the listener's "isHandled(self, obj)"
method (more on the "obj" argument later). If this method returns True,
then the tool calls the "listen(self,obj)" method to handle the journal line.
</p>
<p>
If the tool notices a "quiet" period where no Journal activity occurs, the
tool will call each listener's "noData(self, hadData)" method. Here, hadData
is True if this is the first "noData" call after a period of activity.
</p>
<p>
The object passed to the "isHandled" and "listen" methods is a
Journal.JournalEntry instance. This contains the following interesting
attributes:
</p>
<ul>
<li><strong>lines</strong> List of journal lines related to this entry</li>
<li><strong>fields</strong> List of the database fields parsed out from the lines</li>
<li><strong>didRotate</strong> Boolean, <code>True</code> if
there was a Journal rotation right before this entry</li>
<li><strong>incomplete</strong> Boolean, <code>True</code> if we expect
more lines in this entry</li>
</ul>
<h1>Setting Up SSH For Non-Password Authentication</h1>
<p>
Definitions:
</p>
<pre>
$REMOTE = remote machine to log into
$RUSER = remote user on the remote machine to log in as.
</pre>
<p>
From the machine that will run ssh:
</p>
<pre>
# Create the keys for the local machine
$ ssh-keygen -t rsa
(use default location, and no passphrase)
# Install the ssh keys on the remote machine.
$ scp ~/.ssh/id_rsa.pub $RUSER@$REMOTE:./pubkey
(this will force you to login and authenticate the server,
which must be done at least once)
$ ssh $RUSER@$REMOTE "mkdir ~/.ssh && chmod 700 ~/.ssh ; cat ~/pubkey >> ~/.ssh/authorized_keys ; chmod 600 ~/.ssh/authorized_keys ; rm ~/pubkey"
(this will force you to login again)
$ ssh $RUSER@$REMOTE
(you shouldn't need to log in. If you do, check the local
and remote security files for instructions)
</pre>
<h1>Standard Legaleze</h1>
<p>
The authors are in no way liable for any damage this tool might do. You've
been warned: use at your own risk. Standard administration procedures should
require that you sufficiently test out any new tool before bringing it into
your production environment. Remember to keep good backups. This tool
probably won't eat your lunch, but you still need to be careful.
</p>
</html>
| # | Change | User | Description | Committed | |
|---|---|---|---|---|---|
| #2 | 5810 | Matt Albrecht |
New version of logDaemon that has better bug fixes. Also added server-status web page for a pure client-side AJAX app that pulls together different server-side statistics. |
||
| #1 | 5805 | Matt Albrecht | Update to version 1.3-alpha |