- <html>
- <head>
- <title>logDaemon - A P4 Journal Monitoring Daemon</title>
- </head>
- <body>
- <h1>logDaemon</h1>
- <p>
- A Journal Monitoring Daemon<br/>
- Written for Python 2.3 on *NIX platforms</br>
- Version 1.1</br>
- Placed into the Public Domain
- </p>
- <h1>About</h1>
- <p>
- This tool monitors the live journal file from a Perforce server, and executes a
- chain of registered listeners based on certain events. It is capable of
- correctly handling a journal file being truncated by searching for previous
- journal files (either compressed or not).
- </p>
- <h1>Using The Tool</h1>
- <p>
- The first step is to initialize the Settings.py file. You may either edit the
- Settings.py.template file by hand, or run the CreateSettings.py application
- to do it for you, based on your existing environment variables. This should
- be the only file you need to alter.
- </p>
- <p>
- If you wish to use the hot-backup listener, then you should follow the steps
- below for setting up an SSH communication between the two servers. This
- will mean that both servers should use a minimal-access user for
- communication.
- </p>
- <p>
- Since the tool will look in checkpointed journal files, it is a good idea to
- keep at least the most recent of those around, so that the daemon can
- go back and look up any data it may have missed. If something happens that
- causes the daemon to skip several journal files, including one that was
- deleted, then you will need to manually intervene to correct the situation.
- </p>
- <p>
- Finally, you can extend the tool yourself with custom listeners. See that
- section below.
- </p>
- <h1>Tool Details</h1>
- <p>
- The daemon's main logic is handling the journal file, as it is updated and
- truncated during normal checkpoint operations. The daemon will persist a
- file on the machine running it that describes where it last left off
- (journal file sequence number and line number), so that the daemon can
- correctly recover from either a reboot, abnormal shutdown, or any other
- possible terminating event.
- </p>
- <p>
- This little "where was I" file is very important. It's what ensures that the
- remote system is passed exactly the information it needs. If this file doesn't
- exist, then the tool will look backwards through the journals to find the
- one corresponding to the checkpoint number specified in Setup.py, under
- "SERVER.lastCheckpoint". This number should represent the last checkpoint
- that was recovered on the remote server.
- </p>
- <p>
- If the tool fails to find a journal file in between the current journal and
- either the where-was-I file or the lastCheckpoint, then it will fail. You
- will need to manually recover from this point.
- </p>
- <h1>Correcting From Errors</h1>
- <p>
- The daemon can quit on its own due to several reasons. The most common is
- a mis-configured Setup.py file.
- </p>
- <p>
- Another issue, which can mean serious investigation, is when the daemon
- attempts to find a journal file that is no longer on the host computer.
- In the case of a simple hot-backup usage, you will need to restore the
- remote machine from a recent checkpoint, and update the Setup.py file
- to this checkpoint number. Other tools, such as e-mail reviewing,
- you may want to simply send a message to your users saying that no
- e-mails were sent during this time period, and that they'll have to
- perform a manual monitor themselves.
- </p>
- <h1>Adding Listeners</h1>
- <p>
- The tool parses through the journal files, invoking the registered list of
- listeners on each line encountered.
- </p>
- <p>
- The list of registered listeners is stored in Setup.py, in the JOURNAL
- class. The "listeners" list contains strings of the names of listeners
- (including module name) to load.
- </p>
- <p>
- Each listener is constructed once at daemon start-up time, and should
- extend the listener.Listener class. For each line of the Journal that is
- processed, first the tool invokes the listener's "isHandled(self, obj)"
- method (more on the "obj" argument later). If this method returns True,
- then the tool calls the "listen(self,obj)" method to handle the journal line.
- </p>
- <p>
- If the tool notices a "quiet" period where no Journal activity occurs, the
- tool will call each listener's "noData(self, hadData)" method. Here, hadData
- is True if this is the first "noData" call after a period of activity.
- </p>
- <p>
- The object passed to the "isHandled" and "listen" methods is a
- Journal.JournalEntry instance. This contains the following interesting
- attributes:
- </p>
- <ul>
- <li><strong>lines</strong> List of journal lines related to this entry</li>
- <li><strong>fields</strong> List of the database fields parsed out from the lines</li>
- <li><strong>didRotate</strong> Boolean, <code>True</code> if
- there was a Journal rotation right before this entry</li>
- <li><strong>incomplete</strong> Boolean, <code>True</code> if we expect
- more lines in this entry</li>
- </ul>
- <h1>Setting Up SSH For Non-Password Authentication</h1>
- <p>
- Definitions:
- </p>
- <pre>
- $REMOTE = remote machine to log into
- $RUSER = remote user on the remote machine to log in as.
- </pre>
- <p>
- From the machine that will run ssh:
- </p>
- <pre>
- # Create the keys for the local machine
- $ ssh-keygen -t rsa
- (use default location, and no passphrase)
- # Install the ssh keys on the remote machine.
- $ scp ~/.ssh/id_rsa.pub $RUSER@$REMOTE:./pubkey
- (this will force you to login and authenticate the server,
- which must be done at least once)
- $ ssh $RUSER@$REMOTE "mkdir ~/.ssh && chmod 700 ~/.ssh ; cat ~/pubkey >> ~/.ssh/authorized_keys ; chmod 600 ~/.ssh/authorized_keys ; rm ~/pubkey"
- (this will force you to login again)
- $ ssh $RUSER@$REMOTE
- (you shouldn't need to log in. If you do, check the local
- and remote security files for instructions)
- </pre>
- <h1>Standard Legaleze</h1>
- <p>
- The authors are in no way liable for any damage this tool might do. You've
- been warned: use at your own risk. Standard administration procedures should
- require that you sufficiently test out any new tool before bringing it into
- your production environment. Remember to keep good backups. This tool
- probably won't eat your lunch, but you still need to be careful.
- </p>
- </html>
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#2 | 5810 | Matt Albrecht | New version of logDaemon that has better bug fixes. Also added server-status web p...age for a pure client-side AJAX app that pulls together different server-side statistics. « |
18 years ago | |
#1 | 5805 | Matt Albrecht | Update to version 1.3-alpha | 18 years ago |