<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en"> <head> <title>Perforce Defect Tracking Integration Integrator's Guide</title> </head> <body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000"> <div align="center"> <p><i><a href="http://www.ravenbrook.com/project/p4dti/">Perforce Defect Tracking Integration Project</a></i></p> <hr /> <h1>Perforce Defect Tracking Integration Integrator's Guide</h1> <address> <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>, <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>, 2000-10-16 </address> </div> <h2><a id="contents" name="contents">Contents</a></h2> <ul> <li><a href="#section-1">1. Introduction</a></li> <li><a href="#section-2">2. Understanding the <abbr>P4DTI</abbr></a> <ul> <li><a href="#section-2.1">2.1. Prerequisites</a></li> <li><a href="#section-2.2">2.2. Requirements overview</a></li> <li><a href="#section-2.3">2.3. Architecture overview</a></li> <li><a href="#section-2.4">2.4. Design overview</a></li> </ul> </li> <li><a href="#section-3">3. What you need to do</a> <ul> <li><a href="#section-3.1">3.1. Before you start work</a></li> <li><a href="#section-3.2">3.2. Getting help</a></li> <li><a href="#section-3.3">3.3. Adapting an existing integration</a></li> <li><a href="#section-3.4">3.4. Developing a new integration</a></li> </ul> </li> <li><a href="#section-4">4. Defect tracker database schema extensions</a> <ul> <li><a href="#section-4.1">4.1. Issues</a></li> <li><a href="#section-4.2">4.2. Changelists</a></li> <li><a href="#section-4.3">4.3. Fixes</a></li> <li><a href="#section-4.4">4.4. Filespecs</a></li> <li><a href="#section-4.5">4.5. Replicator configuration and state</a></li> <li><a href="#section-4.6">4.6. Discovering what's changed</a></li> <li><a href="#section-4.7">4.7. Distinguishing replicated changes from other changes</a></li> <li><a href="#section-4.8">4.8. Perforce users who don't have licences in the defect tracker</a></li> </ul> </li> <li><a href="#section-5">5. Coding conventions</a> <ul> <li><a href="#section-5.1">5.1. Messages</a></li> <li><a href="#section-5.2">5.2. Message catalogs</a></li> <li><a href="#section-5.3">5.3. Logging</a></li> <li><a href="#section-5.4">5.4. Errors</a></li> <li><a href="#section-5.5">5.5. Source code layout</a></li> </ul> </li> <li><a href="#section-6">6. The Python interface to the defect tracker</a></li> <li><a href="#section-7">7. The defect tracker module</a> <ul> <li><a href="#section-7.1">7.1. The <code class="source">defect_tracker</code> class</a></li> <li><a href="#section-7.2">7.2. The <code class="source">defect_tracker_issue</code> class</a></li> <li><a href="#section-7.3">7.3. The <code class="source">defect_tracker_fix</code> class</a></li> <li><a href="#section-7.4">7.4. The <code class="source">defect_tracker_filespec</code> class</a></li> <li><a href="#section-7.5">7.5. The <code class="source">translator</code> class</a></li> </ul> </li> <li><a href="#section-8">8. Configuration</a> <ul> <li><a href="#section-8.1">8.1. The configuration generator</a></li> <li><a href="#section-8.2">8.2. Perforce interface configuration</a></li> <li><a href="#section-8.3">8.3. Replicator configuration</a></li> <li><a href="#section-8.4">8.4. Perforce jobspecs</a></li> <li><a href="#section-8.5">8.5. Adapting the configuration module</a></li> <li><a href="#section-8.6">8.6. Making your own configurations</a></li> </ul> </li> <li><a href="#section-9">9. Testing</a></li> <li><a href="#section-10">10. Providing a defect tracker interface to Perforce relations</a></li> <li><a href="#section-11">11. Adapting the manuals</a></li> <li><a href="#section-12">12. Making your work available to the community</a> <ul> <li><a href="#section-12.1">12.1. Reporting defects</a></li> <li><a href="#section-12.2">12.2. Making a contribution</a></li> </ul> </li> <li><a href="#section-A">A. References</a></li> <li><a href="#section-B">B. Document History</a></li> </ul> <h2><a name="section-1" id="section-1">1. Introduction</a></h2> <p> This is the Perforce Defect Tracking Integration (<abbr>P4DTI</abbr>) 1.1 Integrator's Guide. It explains how a developer could extend the <abbr>P4DTI</abbr> to work with defect tracking systems that aren't supported by the standard distribution, or adapt the <abbr>P4DTI</abbr> to work with a supported defect tracker but in some way that isn't supported. </p> <p> The intended readership is developers adapting or extending the <abbr>P4DTI</abbr>, and project staff. </p> <p> This manual is not confidential. </p> <p> I use some words in a precise way to express the importance of an instruction. </p> <ol> <li><p> I say "must" when the instruction is critical. This means that the integration will fail if the instruction is not followed. </p></li> <li><p> I say "should" when the instruction is essential. This means that integration will be of noticeably lower quality than the supported integrations if the instruction is not followed. However, it won't fail. </p></li> <li><p> I say "may" when the instruction is optional. This means that the integration will not suffer much if you don't follow the instruction. </p></li> </ol> <h2><a name="section-2" id="section-2">2. Understanding the <abbr>P4DTI</abbr></a></h2> <p> This section covers the knowledge that you will need in order to extend or adapt the <abbr>P4DTI</abbr>. It gives an overview of the requirements, architecture and design of the <abbr>P4DTI</abbr>, with references to the documents that provide more detail. You must have a good overall understanding of the <abbr>P4DTI</abbr> in order to extend or adapt it. </p> <h3><a name="section-2.1" id="section-2.1">2.1. Prerequisites</a></h3> <p> This manual assumes you are familiar with the following subjects: </p> <ol> <li><p> The jobs subsystem of Perforce, and the relationship between jobs, fixes and changelists [<a href="http://www.perforce.com/perforce/doc.002/manuals/p4guide/10_jobtrack.html#1032869">Perforce 2000-12-22a, 10</a>]. </p></li> <li><p> How the <abbr>P4DTI</abbr> works, from the administrator's point of view. I strongly recommend that you download, install, configure and run one of the supported integrations, following the <cite>Perforce Defect Tracking Integration Administrator's Guide</cite> [<a href="../ag/index.html">RB 2000-08-10a</a>], so that you know what the administrator has to know and do, where the data is stored, what problems can occur. </p></li> <li><p> How the <abbr>P4DTI</abbr> works, from the user's point of view. I strongly recommend that you try out one of the supported integrations, carrying out all the tasks in the <cite>Perforce Defect Tracking Integration User's Guide</cite> [<a href="../ug/index.html">RB 2000-08-10b</a>], so that you know what it's like to use, and what benefit the users get. </p></li> <li><p> The programming language Python. See the <a href="http://www.python.org/">Python web site <http://www.python.org/></a> for downloads and documentation. If you're new to Python, try the tutorial [<a href="http://www.python.org/doc/current/tut/tut.html">van Rossum 2000-10-16</a>], or the book <cite>Programming Python</cite> [<a href="#ref-Lutz-1996">Lutz 1996</a>]. </p></li> </ol> <h3><a name="section-2.2" id="section-2.2">2.2. Requirements overview</a></h3> <p> The five most important requirements are these [<a href="http://www.ravenbrook.com/project/p4dti/req/#req-1">GDR 2000-05-24, 1-5</a>]: </p> <ol> <li><p> Defect tracker state is consistent with the state of the product sources. </p></li> <li><p> The defect tracking integration makes the jobs of the developers and managers easier (i.e. make it easier for them to produce a quality product etc.). </p></li> <li><p> It is easy to discover why the product sources are the way they are, and why they have changed, in terms of the customer requirements. </p></li> <li><p> The interface that allows Perforce to be integrated with defect tracking systems is public, documented, and maintained. </p></li> <li><p> The integration provides the ability to ask questions involving both the defect tracking system and the SCM system. </p></li> </ol> <p> The <abbr>P4DTI</abbr> meets <a href="http://www.ravenbrook.com/project/p4dti/req/#req-1">requirement 1</a> and <a href="http://www.ravenbrook.com/project/p4dti/req/#req-5">requirement 5</a> by replicating data between the defect tracker and Perforce (see <a href="#section-2.3">section 2.3</a>). It meets <a href="http://www.ravenbrook.com/project/p4dti/req/#req-2">requirement 2</a> by making possible for developers to do their routine defect tracking activity entirely from Perforce (by making the defects available through Perforce's jobs interface by replicating them). It meets <a href="http://www.ravenbrook.com/project/p4dti/req/#req-3">requirement 3</a> by supplying a user guide [<a href="../ug/index.html">RB 2000-08-10b</a>] that describes a development process in which issues are linked to changes by making fixes in Perforce. It meets <a href="http://www.ravenbrook.com/project/p4dti/req/#req-4">requirement 4</a> by making the project sources and documents <a href="http://www.ravenbrook.com/project/p4dti/">available to the public</a>. </p> <p> See the <cite>Perforce Defect Tracking Integration Project Requirements</cite> [<a href="http://www.ravenbrook.com/project/p4dti/req/">GDR 2000-05-24</a>] for a full and maintained set of requirements and their original sources. </p> <h3><a name="section-2.3" id="section-2.3">2.3. Architecture overview</a></h3> <p> The <abbr>P4DTI</abbr> meets these requirements using a replication architecture [<a href="../../design/architecture/index.html">RB 2000-08-10c</a>]. A replicator process repeatedly polls two databases (Perforce and the defect tracker) and copies entities from one to the other. This makes and keep them consistent, to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-1">requirement 1</a>; it makes them available to users of both systems, to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-2">requirement 2</a>; and it makes them available for queries combining data from both systems, to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-5">requirement 5</a>. </p> <p> The replicator replicates four relations: </p> <ol> <li><p> <b>Issues</b> are replicated from the defect tracker to Perforce jobs and back again. The replication goes in both directions, but the Perforce jobs should be considered as a subsidiary copy of the real data in the defect tracker. This means that when the two databases differ (for example, because they have been changed simultaneously) the defect tracker is considered to be definitive. </p></li> <li><p> <b>Changelist descriptions</b> are replicated from Perforce to the defect tracker. </p></li> <li><p> <b>Fixes</b> (links between issues and changelists) are replicated in both directions. </p></li> <li> <p> <b>Filespecs</b> (links between issues and files) are replicated in both directions. </p> </li> </ol> <p> The <abbr>P4DTI</abbr> replicates the filespec relation in order to support use cases like "Associate revisions of documents with task" [<a href="http://www.ravenbrook.com/project/p4dti/doc/2000-05-03/reqs-and-use-cases/">GDR 2000-05-03, 6.2</a>] and "Check out copies of revisions of documents associated with task" [<a href="http://www.ravenbrook.com/project/p4dti/doc/2000-05-03/reqs-and-use-cases/">GDR 2000-05-03, 6.3</a>] and to support defect trackers like DevTrack by TechExcel that provide a revision control interface based on associating documents with an issue. However, because the supported defect trackers (TeamTrack and Bugzilla) have no such feature, and because alpha and beta testing showed no demand for use cases involving associating documents with tasks, we haven't made any use of this relation (for example, it's not documented in the user's guide). However, it's there if you need it for integrating with your defect tracker. </p> <h3><a name="section-2.4" id="section-2.4">2.4. Design overview</a></h3> <p> The replicator is designed to be highly independent of both Perforce and the defect tracker, using public interfaces wherever possible, so that the integration doesn't have to change frequently to keep up with the systems it integrates (<a href="http://www.ravenbrook.com/project/p4dti/req/#req-27">requirement 27</a>) and the cost of maintenance is low (<a href="http://www.ravenbrook.com/project/p4dti/req/#req-30">requirement 30</a>). It runs as a separate process and uses public protocols to access both databases. It doesn't require any special support from either system (though users will benefit if the defect tracker provides an interface to Perforce fixes; (see <a href="#section-10">section 10</a>). </p> <p> The replicator is written in the interpreted programming language Python, a portable, stable, readable and open programming language (to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-21">requirement 21</a>, <a href="http://www.ravenbrook.com/project/p4dti/req/#req-24">requirement 24</a>, <a href="http://www.ravenbrook.com/project/p4dti/req/#req-25">requirement 25</a>, and <a href="http://www.ravenbrook.com/project/p4dti/req/#req-26">requirement 26</a>.). </p> <p> <a href="#figure-1">Figure 1</a> below shows the broad outlines of how the replicator is constructed. Parts in black are shared by the integrations with all defect trackers. The components in red are the components that you must write in order to integrate with your defect tracker. </p> <p> If you need to modify any other components to integrate with your defect tracker, that's a defect in the integration kit. Please report it (see <a href="#section-12.1">section 12.1</a>) or make the necessary modifications and submit them as a contribution (see <a href="#section-12.2">section 12.2</a>). </p> <div align="center"> <p> <a id="figure-1" name="figure-1">Figure 1</a>. Replicator block diagram </p> <img src="replicator-design.gif" alt="Replicator block diagram" width="444" height="467" /> </div> <p> The replicator is divided into these components: </p> <ol> <li><p> <a href="../../code/replicator/replicator.py"><code class="filename">replicator.py</code></a>, a module that runs the replication algorithm and reports failures by e-mail [<a href="../../design/replicator/index.html">GDR 2000-09-13</a>]. </p></li> <li><p> A defect tracker module, called <code class="filename">dt_<i>defect_tracker</i>.py</code> (<code class="filename">dt_abc.py</code> in <a href="#figure-1">figure 1</a>). This defines a subclass of <code class="filename">defect_tracker</code>. It is responsible for fetching, translating and updating entities in the defect tracker database at a high level of abstraction (see <a href="#section-7">section 7</a>). For each new defect tracker, you need to write such a module. </p></li> <li><p> A defect tracker interface. This provides a low-level interface to the defect tracker, allowing entities to be fetched from and stored in records in the defect tracker's database (see <a href="#section-6">section 6</a>). For each new defect tracker, you need to write such a module. </p></li> <li><p> A configuration generator. This is responsible for taking the raw configuration supplied by the <abbr>P4DTI</abbr> administrator and building a detailed configuration suitable for the replicator (see <a href="#section-8">section 8</a>). </p></li> <li><p> Supporting components, including an interface to Perforce, a message system, and configuration checking functions. </p></li> </ol> <h2><a name="section-3" id="section-3">3. What you need to do</a></h2> <p> This section gives an overview of the work required in adapting an existing integration or developing a new integration. </p> <h3><a name="section-3.1" id="section-3.1">3.1. Before you start work</a></h3> <p> Someone might already have developed the integration or adaption that you plan to work on. Take a look at the <a href="http://www.ravenbrook.com/project/p4dti/contrib/"><abbr>P4DTI</abbr> contributions page <http://www.ravenbrook.com/project/p4dti/contrib/></a>. </p> <p> Someone might be currently be working on the integration or adaption that you plan to work on. If so, <a href="http://www.perforce.com/perforce/support.html">Perforce support</a> may know about them. </p> <p> The feature you want may in fact be part of the supported <abbr>P4DTI</abbr> product and it is missing from the manuals or the manuals unclear. If so, <a href="http://www.perforce.com/perforce/support.html">Perforce support</a> can tell you. And if the manuals are unclear or missing information, then please submit a defect report (see <a href="#section-12.1">section 12.1</a>). </p> <h3><a name="section-3.2" id="section-3.2">3.2. Getting help</a></h3> <p> The Perforce Defect Tracking Integration Kit is a supported product. If you have trouble adapting the <abbr>P4DTI</abbr> or developing an integration after following the instructions in here, you should contact <a href="http://www.perforce.com/perforce/support.html">Perforce support</a> for help. </p> <p> <a href="http://www.ravenbrook.com/services/perforce/">Ravenbrook Limited</a> may be able to develop or consult on adaptions and extensions to the P4DTI. </p> <h3><a name="section-3.3" id="section-3.3">3.3. Adapting an existing integration</a></h3> <p> This section gives an overview of what you should do if you want to adapt the <abbr>P4DTI</abbr> to work with a supported defect tracker but in some way that isn't supported. For example: </p> <ol> <li> You want the replicator to leave your Perforce jobspec unchanged. </li> <li> You want more control over which defect tracker issues are replicated to Perforce. </li> <li> You want to change the names of the fields in Perforce. </li> <li> You want to match up the fields in an unusual way, for example you have a single field in the defect tracker that ought to be two fields in Perforce (or vice versa). </li> <li> You want to translate values in a field in a different way. </li> <li> You want to use a locally customized version of Bugzilla. </li> </ol> <p> In these and many similar cases, you should make the <abbr>P4DTI</abbr> do what you want by writing a "configuration generator" (see <a href="#section-8.6">section 8.6</a>). </p> <p> But don't skip straight to that section. At least skim the rest of the manual. You'll need to understand many of the details in order to write a configuration generator, especially how to write translator classes (see <a href="#section-7.5">section 7.5</a>) and how the configuration works (see <a href="#section-8">section 8</a>). </p> <h3><a name="section-3.4" id="section-3.4">3.4. Developing a new integration</a></h3> <p> This section gives an outline of the steps you need to take to integrate Perforce with a new defect tracker. </p> <ol> <li><p> You must choose a name for the integration. This should be the name of the defect tracker, for example "TeamTrack" or "Bugzilla". This name (when converted to lower case) must be used as part of the names of modules making up the integration (see <a href="#section-7">section 7</a> and <a href="#section-8">section 8</a>). </p></li> <li> <p> You must provide full implementations of these components: </p> <ol> <li><p> A documented design for extensions for the defect tracker database schema (see <a href="#section-4">section 4</a>); </p></li> <li><p> A Python interface to the defect tracker (see <a href="#section-6">section 6</a>); </p></li> <li><p> A defect tracker module (see <a href="#section-7">section 7</a>); </p></li> <li><p> A configuration generator (see <a href="#section-8">section 8</a>). </p></li> </ol> </li> <li><p> You should develop and apply tests (both automated and manual) of your integration (see <a href="#section-9">section 9</a>). </p> </li> <li><p> You should provide a defect tracker interface to the Perforce relations, if possible (see <a href="#section-10">section 10</a>).</p></li> <li><p> You must adapt or extend these components: </p> <ol> <li><p> The configuration module <a href="../../code/replicator/config.py"><code class="filename">config.py</code></a> (see <a href="#section-8.5">section 8.5</a>). </p></li> <li><p> The Administrator's Guide (see <a href="#section-11">section 11</a>); </p></li> <li><p> The User's Guide (see <a href="#section-11">section 11</a>); </p></li> </ol> <p> All other components are designed to be portable between defect trackers. If your integration cannot be made to work without changing the portable components, then there is a defect in the P4DTI Integration Kit. Please report this (see <a href="#section-12.1">section 12.1</a>). </p> </li> <li><p> Once all the work outlined above is completed and tested to your satisfaction, you should make your work available to the community so that others can benefit from your efforts (see <a href="#section-12.2">section 12.2</a>). </p></li> </ol> <p> I estimate that at least 10 weeks of effort are required to develop, test, document and release a new integration [<a href="/project/p4dti/doc/2000-05-30/arch-analysis/#section-2.21.1">GDR 2000-05-30</a>] </p> <h2><a name="section-4" id="section-4">4. Defect tracker database schema extensions</a></h2> <p> You must extend the database schema by adding new fields to the issue relation (see <a href="#section-4.1">section 4.1</a>), and adding three new relations: the changelist relation (see <a href="#section-4.2">section 4.2</a>), the fixes relation (see <a href="#section-4.3">section 4.3</a>), and the filespecs relation (see <a href="#section-4.4">section 4.4</a>). You should add another relation to the database, to store the replicator state and configuration (see <a href="#section-4.5">section 4.5</a>). These schema extensions must be documented so that users of your integration can implement database queries and reports that use this data, to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-5">requirement 5</a>. </p> <p> These relations should be stored in separate tables if possible, to most easily support queries and reporting using standard database tools. However, some defect trackers do not support this. </p> <blockquote><p> <i>Example.</i> TeamTrack release 4.5 doesn't support the addition of tables to its database schema, so the TeamTrack schema extensions squash these relations into a single table, using a type field to distinguish them [<a href="../../design/teamtrack-p4dti-schema/index.html#section-2.1">GDR 2000-09-04, 2.1</a>]. </p></blockquote> <p> The design must support multiple replicators replicating from a single defect tracker, and support a single replicator replicating to multiple Perforce servers from one defect tracker, in order to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-96">requirement 96</a>. To support this, each relation includes a replicator identifier which identifies the replicator which is handling replication for that record, and a Perforce server identifier, which identifies the Perforce server that the record is replicated to. </p> <blockquote><p> <i>Examples.</i> The TeamTrack database schema extensions [<a href="../../design/teamtrack-p4dti-schema/index.html">GDR 2000-09-04</a>] and the Bugzilla database schema extensions [<a href="../../design/bugzilla-p4dti-schema/index.html">NB 2000-11-14b</a>]. </p></blockquote> <h3><a name="section-4.1" id="section-4.1">4.1. Issues</a></h3> <p> The issue relation must be extended with these fields: </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top" align="left"> <th>Field contents</th> <th>Field type</th> </tr> <tr valign="top"> <td>Replicator identifier of the replicator that is in charge of replicating this issue, or the empty string or NULL if the issue is not replicated.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Server identifier of the Perforce server to which this issue is replicated, or the empty string or NULL if the issue is not replicated.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Name of Perforce job to which this issue is replicated, or the empty string or NULL if the issue is not replicated.</td> <td><code class="source">char(1024)</code> (or <code class="source">varchar(1024)</code> since most jobnames are short).</td> </tr> </table> <p> You may add these fields to the defect tracker's issue table, or you may store them in a separate table and use the issue key to relate the two tables. </p> <blockquote><p> <i>Examples.</i> The TeamTrack integration adds the new fields to the existing <code class="source">TS_CASES</code> table [<a href="../../design/teamtrack-p4dti-schema/index.html#section-3.1">GDR 2000-09-04, 3.1</a>]. The Bugzilla integration creates a table <code class="source">p4dti_bugs</code> containing the new fields and associates them with the <code class="source">bugs</code> table using the <code class="source">bug_id</code> field [<a href="../../design/bugzilla-p4dti-schema/index.html">NB 2000-11-14b</a>]. </p></blockquote> <h3><a name="section-4.2" id="section-4.2">4.2. Changelists</a></h3> <p> The changelist relation has these fields: </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top" align="left"> <th>Field contents</th> <th>Field type</th> </tr> <tr valign="top"> <td>Replicator identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Perforce server identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Change number.</td> <td><code class="source">int</code></td> </tr> <tr valign="top"> <td>User who created the change.</td> <td>A foreign key reference to the defect tracker's user relation giving the user who created or submitted the change.</td> </tr> <tr valign="top"> <td>Change status.</td> <td>An enumeration with two values: pending or submitted.</td> </tr> <tr valign="top"> <td>Date the change was last modified.</td> <td>A date and time.</td> </tr> <tr valign="top"> <td>Change description.</td> <td>Text, unlimited in length.</td> </tr> <tr valign="top"> <td>Client from which the change was submitted.</td> <td><code class="source">char(1024)</code> (or <code class="source">varchar(1024)</code> since most client names are short).</td> </tr> </table> <p> The combination of (change number, Perforce server identifier) is the primary key for this relation: there can only be one change with a particular number on a Perforce server. </p> <h3><a name="section-4.3" id="section-4.3">4.3. Fixes</a></h3> <p> The fixes relation has these fields: </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top" align="left"> <th>Field contents</th> <th>Field type</th> </tr> <tr valign="top"> <td>Replicator identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Perforce server identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Issue.</td> <td>A foreign key reference to the defect tracker's issue relation, giving the issue which is fixed by the change.</td> </tr> <tr valign="top"> <td>Change number</td> <td><code class="source">int</code></td> </tr> <tr valign="top"> <td>Date the fix was last modified.</td> <td>A date and time.</td> </tr> <tr valign="top"> <td>User who created the fix.</td> <td>A foreign key reference to the defect tracker's user relation, giving the user who last modified the fix.</td> </tr> <tr valign="top"> <td>Status the job was/will be fixed to.</td> <td><code class="source">char(1024)</code> (or <code class="source">varchar(1024)</code> since most job statuses are short).</td> </tr> <tr valign="top"> <td>Client from which the fix was made.</td> <td><code class="source">char(1024)</code> (or <code class="source">varchar(1024)</code> since most client names are short).</td> </tr> </table> <p> The combination of (issue, change number, Perforce server identifier) is the primary key for this relation: there can only be one fix between a change and an issue on a Perforce server. </p> <h3><a name="section-4.4" id="section-4.4">4.4. Filespecs</a></h3> <p> The associated filespecs relation has these fields: </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top" align="left"> <th>Field contents</th> <th>Field type</th> </tr> <tr valign="top"> <td>Replicator identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Perforce server identifier.</td> <td><code class="source">char(32)</code></td> </tr> <tr valign="top"> <td>Issue.</td> <td>A foreign key reference to the defect tracker's issue relation, giving the issue which is fixed by the change.</td> </tr> <tr valign="top"> <td>Filespec.</td> <td><code class="source">text</code></td> </tr> </table> <h3><a name="section-4.5" id="section-4.5">4.5. Replicator configuration and state</a></h3> <p> By design, the replicator has no internal state. This is to make the replicator robust against losing a network connection, or the machine it's running on crashing in the middle of a replication: when the network comes back up or it starts again, it tries the replication again [<a href="../../design/replicator/index.html#section-2.9">GDR 2000-09-13, 2.9</a>]. This design principle helps to meet <a href="/project/p4dti/req/#req-1">requirement 1</a> (consistency between databases). </p> <p> This means that if you need to store information, such as a record of which changes have been replicated (see <a href="#section-4.6">section 4.6</a>) you must store it in the defect tracker's database. </p> <p> The replicator also needs to pass information to the defect tracker, to support an interface from the defect tracker to Perforce (see <a href="#section-10">section 10</a>). There are two configuration parameters which should be communicated to the defect tracker by storing them in a configuration table: <a href="../ag/index.html#config-changelist_url"><code class="source">changelist_url</code></a> and <a href="../ag/index.html#config-p4_server_description"><code class="source">p4_server_description</code></a>. </p> <h3><a name="section-4.6" id="section-4.6">4.6. Discovering what's changed</a></h3> <p> The replicator works by repeatedly polling the databases, so you must provide a way to tell it which issues have changed since the last time it polled. Here are some strategies: </p> <ol> <li> <p> If the defect tracker has a changes table which records the history of changes to issues, then store a record number in the replicator state that gives the last record in the changes table that has been replicated.</p> <blockquote><p> <i>Example.</i> The TeamTrack integration uses this approach [<a href="../../design/teamtrack-p4dti-schema/index.html#section-3.5">GDR 2000-09-04, 3.5</a>]. </p></blockquote> </li> <li><p> If the defect tracker has a last modified date field in the issue table, store the value of this field at the point when the replicator was last replicated. Then you can fetch the changed issues by looking for issues whose last modified date is greater than the last replicated date. This is likely to be less efficient than solution 1. </p></li> <li><p> Modify the defect tracker so that it supports solution 1 or 2. </p></li> <li><p> If all else fails, you could store a "shadow" table of issues, containing copies of the issue records as they were when last modified. Then you can find changed issues by finding differing corresponding records. This is likely to be very inefficient. </p></li> </ol> <h3><a name="section-4.7" id="section-4.7">4.7. Distinguishing replicated changes from other changes</a></h3> <p> The replicator needs to distinguish the changes it made from changes made by other users of the defect tracker. Otherwise it will attempt to replicate its own changes back to Perforce. This won't actually end up in an infinite loop of replication, since when it replicates back it will discover that there are no changes to be made, and so not actually do anything. However, this double replication gives twice the opportunity for conflicts, and hence annoying e-mail messages for the users of the <abbr>P4DTI</abbr> (see <a href="http://www.ravenbrook.com/project/p4dti/issue/job000016/">Ravenbrook issue job000016</a>). </p> <p> Here are some strategies: </p> <ol> <li> <p> Suppose that the defect tracker has separate concepts of "logged in user" and "user who is making the change". In this case, make a special user to represent the replicator and have the replicator log in as that user. The replicator's changes show up with logged in user being the replicator user; all other changes need to be replicated. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration uses this approach [<a href="../../design/teamtrack-p4dti-schema/index.html#section-5">GDR 2000-09-04, 5</a>]. </p></blockquote> </li> <li> <p> You could store a table listing the changes that were made by the replicator. Any other changes need to be replicated. </p> </li> <li><p> If the defect tracker has a last modified date field in the issue table, store the value of this field at the point when the replicator was last replicated. Then an issue has been changed by someone else if its last modified date differs from the last replicated date. </p></li> </ol> <h3><a name="section-4.8" id="section-4.8">4.8. Perforce users who don't have licences in the defect tracker</a></h3> <p> The replicator replicates user fields in issues, changelists and fixes (for example, the owner of an issue or the user who submitted a changelist) by applying a user translation function (see <a href="#section-7.5.4">section 7.5.4</a>). When a defect tracker user has no licence in Perforce, the translation function can simply use that user's defect tracker login name, since Perforce doesn't validate user fields in jobs. But if a Perforce user has no licence in the defect tracker, the translator needs to do something with them. For issues your defect tracker interface should simply refuse to replicate when a Perforce user has no licence in the defect tracker. But you should be less strict when replicating fixes and changelists: while it is a sensible policy (and required by some defect tracker vendors) to require the current owner of a job in Perforce to have a licence, it is not sensible to require every user in Perforce who ever submitted a changelist to have a licence in the defect tracker. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration maps unknown users in changelists and fixes to the special TeamTrack user 0 (representing "no user"). When there's an unknown user in an issue, the integration rejects the attempt to replicate it by raising an error. </p></blockquote> <h2><a name="section-5" id="section-5">5. Coding conventions</a></h2> <p> This section covers coding conventions followed in the <abbr>P4DTI</abbr>. You should follow these conventions in your adaptions and extensions. They will make your code more reliable and easier to debug. It will be easier for users to diagnose problems and fix them. If you contribute your code for inclusion in the <abbr>P4DTI</abbr> (see <a href="#section-12.2">section 12.2</a>) then it will be easier for us to integrate your contribution. </p> <blockquote><p> <i>Examples.</i> Look at the TeamTrack module, <a href="../../code/replicator/dt_teamtrack.py"><code class="filename">dt_teamtrack.py</code></a>, and the Bugzilla module, <a href="../../code/replicator/dt_bugzilla.py"><code class="filename">dt_bugzilla.py</code></a> for uses of all conventions and features covered in this section. </p></blockquote> <h3><a name="section-5.1" id="section-5.1">5.1. Messages</a></h3> <p> The <a href="../../code/replicator/message.py"><code class="filename">message.py</code></a> module defines a class of messages. You must use this class when writing messages to the replicator's log (see <a href="#section-5.3">section 5.3</a>). You should use this class when raising errors (see <a href="#section-5.4">section 5.4</a>). </p> <p> You create a message like this: </p> <blockquote><code class="source"> import message <br /> id = 123 <br /> text = "Constructed a test message." <br /> priority = message.DEBUG <br /> product = "Test" <br /> msg = message.message(id, text, priority, product) </code></blockquote> <p> The four arguments to the constructor are as follows: </p> <ol> <li><p> The <code class="source">id</code> argument is a message identifier (an integer). This must be unique among all messages generated by the product. </p></li> <li><p> The <code class="source">text</code> argument is the text of the message (a string). </p></li> <li><p> The <code class="source">priority</code> argument is the level of importance of the message. This must be one of the constants in the following table: </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top"> <td> <code class="source">message.CRIT</code> </td> <td> A critical error: the replicator will stop immediately. Use this priority for errors in configuration discovered by your configuration generator or by the <a href="#defect_tracker.__init__"><code class="source">__init__()</code></a> and <a href="#defect_tracker.init"><code class="source">init()</code></a> methods of your defect tracker interface. </td> </tr> <tr valign="top"> <td> <code class="source">message.ERR</code> </td> <td> An error. The replicator can't complete some operation. Use this priority for errors discovered during replication, such as untranslatable fields or permission failures. </td> </tr> <tr valign="top"> <td> <code class="source">message.WARNING</code> </td> <td> <p> A warning. The replicator can continue, but the administrator may want to take some action. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration issues a warning when it finds two entities with the same name in the database (for example, two projects, or two resolutions). The integration can continue (it just uses the first entity it finds), but it may not do what users expect. </p></blockquote> </td> </tr> <tr valign="top"> <td> <code class="source">message.NOTICE</code> </td> <td> <p> A significant but expected condition. </p> <blockquote><p> <i>Example.</i> The replicator uses this priority when it has to overwrite a Perforce job with a defect tracker issue. This is the correct thing to do, and is the documented behaviour [<a href="../ug/index.html#section-2.2">UG, 2.2</a>] but it is still a significant event. </p></blockquote> </td> </tr> <tr valign="top"> <td> <code class="source">message.INFO</code> </td> <td> For information only. </td> </tr> <tr valign="top"> <td> <code class="source">message.DEBUG</code> </td> <td> Unlikely to be useful except for debugging. </td> </tr> </table> </li> <li><p> The <code class="source">product</code> argument is the name of software product which generated the message. For the supported <abbr>P4DTI</abbr>, this must be <code class="source">"P4DTI"</code>. </p></li> </ol> <p> You can format a message as text by converting the message object to a string: </p> <blockquote><code class="source"> >>> str(msg) <br /> "(Test-123X) Constructed a test message." </code></blockquote> <p> Note that a check digit has been appended to the message identifier. (The check digit uses a mod-11 algorithm similar to that used in ISBNs, so the check digit can be 0-9 or X.) The idea of the check digit is so that Perforce support can ask users for the message identifier of the error that they are reporting. The check digit makes it very likely likely that if the error is misreported or misheard the problem will be detected. </p> <p> You can wrap a message to some number of columns by calling its <code class="source">wrap()</code> method: </p> <blockquote><code class="source"> >>> print msg.wrap(25) <br /> (Test-123X) Constructed <br /> a test message. </code></blockquote> <h3><a name="section-5.2" id="section-5.2">5.2. Message catalogs</a></h3> <p> You may create each message when you need it, but you should use a message catalog. A catalog helps you keep message identifiers distinct and internationalizes your code. </p> <p> A message catalog is a dictionary that maps message identifier to a tuple of two elements: the message priority, and a format string that can be used to build the message text. For example: </p> <blockquote><code class="source"> # Test catalog in English <br /> test_en_catalog = { <br /> 123: (message.DEBUG, "Constructed a test message."), <br /> 124: (message.CRIT, "Couldn't connect to defect tracker on host '%s'."), <br /> 125: (message.ERR, "User '%s' has no permission to edit issue '%s'."), <br /> 126: (message.INFO, "Replicated issue %d."), <br /> # ...<br /> } </code></blockquote> <p> Once you have a message catalog for a product, you should build a message factory that will dispense messages from that catalog, like this: </p> <blockquote><code class="source"> import message <br /> product = "Test" <br /> factory = message.catalog_factory(test_en_catalog, product) </code></blockquote> <p> Now you can construct a message by calling the factory's <code class="source">new()</code> method and passing the message identifier, and the arguments for the format string: </p> <blockquote><code class="source"> msg1 = factory.new(124, 'dt.ravenbrook.com') <br /> msg2 = factory.new(125, ('gdr', 'BUG00123')) </code></blockquote> <p> See the <a href="../../code/replicator/catalog.py"><code class="filename">catalog.py</code></a> module for the <abbr>P4DTI</abbr> catalog and message factory. </p> <h3><a name="section-5.3" id="section-5.3">5.3. Logging</a></h3> <p> The <abbr>P4DTI</abbr> logs its progress and errors by creating messages (see <a href="#section-5.1">section 5.1</a>) and sending them to a "logger": that is, an instance of the logger class defined by the <a href="../../code/replicator/logger.py"><code class="filename">logger.py</code></a> module. </p> <p> The logger module defines classes for logging to files, to standard output, and to the system log on Unix. The <code class="source">multi_logger</code> class directs a single message to several loggers. Each logger class takes a priority argument on instantiation: only messages with this priority, or a higher priority, will appear in the log. </p> <p> You should log as many debugging messages as you like (by default the <a href="../ag/index.html#config-log_level"><code class="source">log_level</code></a> configuration parameter is <code class="source">message.INFO</code> so these messages won't appear). You should log informational messages sparingly, and only when you actually make a change in a database. You should not log error messages, but should raise them as exceptions instead (see <a href="#section-5.4">section 5.4</a>); the replicator will log them for you when it catches them. </p> <p> To add a message to a log, create a message object (see <a href="#section-5.1">section 5.1</a>) and pass it to the logger's <code class="source">log()</code> message: </p> <blockquote><code class="source"> import logger <br /> # Log messages of priority INFO and higher to test.log: <br /> logger_object = logger.file_logger("test.log", message.INFO) <br /> msg = factory.new(126, issue_id) <br /> logger_object.log(msg) </code></blockquote> <p> The configuration generator (see <a href="#section-8">section 8</a>) must construct a logger object for use by the replicator. The same logger object should be used by the defect tracker module (see <a href="#section-7">section 7</a>) as well, so that all messages are collected in the same place. You must allow the <abbr>P4DTI</abbr> administrator to control the volume of log messages by setting the <a href="../ag/index.html#config-log_level"><code class="source">log_level</code></a> configuration parameter. </p> <h3><a name="section-5.4" id="section-5.4">5.4. Errors</a></h3> <p> In the <abbr>P4DTI</abbr>, errors are indicated by raising a Python exception, not by returning an exceptional value. </p> <p> Raise an error using a string as the exception object, and a message object (see <a href="#section-5.1">section 5.1</a>) as the message. For example: </p> <blockquote><code class="source"> error = "Example error" <br /> # ... <br /> raise error, factory.new(124, 'dt.ravenbrook.com') </code></blockquote> <p> It doesn't make any difference what priority you give to the message, but it is conventional to use <code class="source">message.CRIT</code> when the replicator will stop (for example, configuration errors), and <code class="source">message.ERR</code> when the replicator will continue (for example, untranslatable fields or permission failures). </p> <h3><a name="section-5.5" id="section-5.5">5.5. Source code layout</a></h3> <p> You should include in each file of source code: </p> <ol> <li><p> The author. </p></li> <li><p> An introduction explaining what the file is intended to achieve: for example, which requirements does it help to meet? </p></li> <li><p> A references section listing the sources you've used in preparing the code and which other people should read in order to understand it. You should certainly refer to the appropriate sections in this manual. </p></li> <li><p> A history section listing the changes made to the code, with the date and the person who change it. </p></li> <li><p> A statement of copyright. </p></li> <li><p> A licence giving people permission to copy the file under certain conditions (or denying them permission if that's what you intend). </p></li> </ol> <blockquote><p> <i>Example.</i> The <a href="../../code/replicator/logger.py"><code class="filename">logger.py</code></a> module displays all these features. </p></blockquote> <h2><a name="section-6" id="section-6">6. The Python interface to the defect tracker</a></h2> <p> You'll need a way for Python to read and write defect tracking records. If the defect tracker has an API of some sort, you'll need to use that; if not, you'll have to read and write the database directly, using one of the Python database interfaces. Your defect tracker interface will need to support these kinds of operations: </p> <ol> <li><p> Get an issue record. </p></li> <li><p> Update an issue record. </p></li> <li><p> Get all the issues needing replication. </p></li> <li><p> Get all the fixes for an issue. </p></li> <li><p> Add/update/delete a fix. </p></li> <li><p> Create a table. </p></li> <li><p> Add a field to a table. </p></li> <li><p> Get a list of the fields that make up the issue relation, together with the field types, lengths, legal values, etc. </p></li> <li><p> Get a list of users, with names, userids, e-mail addresses. </p></li> </ol> <p> I can't give you a complete or precise list of operations here; you'll have to see what's required as you implement your schema extensions (see <a href="#section-4">section 4</a>) and defect tracker module (see <a href="#section-7">section 7</a>). </p> <blockquote><p> <i>Example.</i> The TeamTrack integration uses the TeamShare API to connect to the defect tracker, because the API provides methods that apply TeamTrack's privilege system and database validation. The integration uses a Python extension module that provides an interface to the parts of the TeamShare API that it needs (only a small part of the whole API, as it happened). See the <a href="../../code/python-teamtrack-interface/index.html">Python interface to TeamTrack</a> and its design [<a href="../../design/python-teamtrack-interface/index.html">GDR 2000-08-08</a>]. </p></blockquote> <blockquote><p> <i>Example.</i> Bugzilla has no API: you have to understand the Bugzilla database schema [<a href="../../design/bugzilla-schema/index.html">NB 2000-11-14a</a>] and connect directly to the MySQL database. The Bugzilla integration uses a wrapper module that encapsulates the direct database operations as defect tracker oriented functions like <code class="source">update_bug</code>. See <a href="../../code/replicator/bugzilla.py"><code class="filename">bugzilla.py</code></a> and its design [<a href="../../design/python-bugzilla-interface/index.html">NB 2000-11-14c</a>]. </p></blockquote> <h2><a name="section-7" id="section-7">7. The defect tracker module</a></h2> <p> You must create a module called <code class="filename">dt_<i>defect_tracker</i>.py</code> (where <code class="filename"><i>defect_tracker</i></code> is the lower-case form of the name your chose for your defect tracker (see <a href="#section-3">section 3</a>)) that implements these classes: </p> <ol> <li><p> The defect tracker interface itself: a subclass of <code class="source">dt_interface.defect_tracker</code> (see <a href="#section-7.1">section 7.1</a>). </p></li> <li><p> Defect tracker issue: a subclass of <code class="source">dt_interface.defect_tracker_issue</code> (see <a href="#section-7.2">section 7.2</a>). </p></li> <li><p> Defect tracker fix: a subclass of <code class="source">dt_interface.defect_tracker_fix</code> (see <a href="#section-7.3">section 7.3</a>). </p></li> <li><p> Defect tracker filespec: a subclass of <code class="source">dt_interface.defect_tracker_filespec</code> (see <a href="#section-7.4">section 7.4</a>). </p></li> <li><p> A translator between dates in the defect tracker and Perforce: a subclass of <code class="source">translator.translator</code> (see <a href="#section-7.5.1">section 7.5.1</a>). </p></li> <li><p> A translator between multi-line text fields in the defect tracker and Perforce: a subclass of <code class="source">translator.translator</code> (see <a href="#section-7.5.3">section 7.5.3</a>). </p></li> <li><p> A translator between users in the defect tracker and Perforce: a subclass of <code class="source">translator.user_translator</code> (see <a href="#section-7.5.4">section 7.5.4</a>). </p></li> <li><p> Any other translator classes that will be needed to translate fields in the issue relation (see <a href="#section-7.5">section 7.5</a>). </p></li> </ol> <blockquote><p> <i>Examples.</i> The TeamTrack module, <a href="../../code/replicator/dt_teamtrack.py"><code class="filename">dt_teamtrack.py</code></a>, and the Bugzilla module, <a href="../../code/replicator/dt_bugzilla.py"><code class="filename">dt_bugzilla.py</code></a>. </p></blockquote> <h3><a name="section-7.1" id="section-7.1">7.1. The <code class="source">dt_interface.defect_tracker</code> class</a></h3> <p> A subclass of <code class="source">dt_interface.defect_tracker</code> implements the replicator's interface to a defect tracker. </p> <blockquote><p> <i>Examples.</i> The <a href="../../code/replicator/dt_teamtrack.py"><code class="source">dt_teamtrack.py</code></a> module defines a class <code class="source">dt_teamtrack</code>. The <a href="../../code/replicator/dt_bugzilla.py"><code class="source">dt_bugzilla.py</code></a> module defines a class <code class="source">dt_bugzilla</code>. </p></blockquote> <p> A subclass of <code class="source">dt_interface.defect_tracker</code> must define the following methods: </p> <h4> <a name="defect_tracker.__init__" id="defect_tracker.__init__"><code class="source">__init__(self, config)</code></a> </h4> <p> This is called when the defect tracker object is created. The <code class="source">config</code> parameter is an object whose attributes are the configuration parameters for the defect tracker. See <a href="#section-8">section 8</a> for the details of how configuration parameters end up in this object. </p> <p> This method should check that all configuration parameters are supplied and have valid values. Use the methods in <code class="filename"><a href="../../code/replicator/check_config.py">check_config.py</a></code> for basic checks. </p> <p> The required parameters should certainly include <a href="../ag/index.html#config-changelist_url"><code class="source">changelist_url</code></a>, <a href="../ag/index.html#config-p4_server_description"><code class="source">p4_server_description</code></a>, <a href="../ag/index.html#config-rid"><code class="source">rid</code></a>, <a href="../ag/index.html#config-sid"><code class="source">sid</code></a>, and <a href="../ag/index.html#config-start_date"><code class="source">start_date</code></a>, but may include others, either supplied by the <abbr>P4DTI</abbr> administrator in <a href="../../code/replicator/config.py"><code class="filename">config.py</code></a> or generated by the configuration generator. </p> <h4> <a name="defect_tracker.all_issues" id="defect_tracker.all_issues"><code class="source">all_issues(self)</code></a> </h4> <p> Return a list of all defect tracking issues that have been modified since the starting point for replication (that is, the date given by the <abbr>P4DTI</abbr> administrator in the <a href="../ag/index.html#config-start_date"><code class="source">start_date</code></a> parameter). </p> <p> Include in this list: </p> <ol> <li><p> Issues replicated by this replicator (that is, the replicator identifier for those issues matches the <a href="../ag/index.html#config-rid"><code class="source">rid</code></a> configuration parameter); </p></li> <li><p> Issues not replicated by any replicator (that is, the replicator identifier for those issues is blank). </p></li> </ol> <p> Omit from this list: </p> <ol> <li><p> Issues replicated by a different replicator (that is, the replicator identifier for those issues differs from the <a href="../ag/index.html#config-rid"><code class="source">rid</code></a> configuration parameter); </p></li> <li><p> Issues unchanged since the start date. </p></li> </ol> <p> Each element of the returned list must belong to your subclass of the <code class="source">dt_interface.defect_tracker_issue</code> class (see <a href="#section-7.2">section 7.2</a>). </p> <h4> <a name="defect_tracker.changed_entities" id="defect_tracker.changed_entities"><code class="source">changed_entities(self)</code></a> </h4> <p> This method is called at the start of each replication cycle to determine what work there is to do. </p> <p> It must return a tuple of three elements: </p> <ol> <li> <p> A list of defect tracking issues that require replication. </p> <p> Each element of the returned list must belong to your subclass of the <code class="source">dt_interface.defect_tracker_issue</code> class (see <a href="#section-7.2">section 7.2</a>). </p> <p> Include in this list: </p> <ol> <li><p> Issues replicated by this replicator (that is, the replicator identifier for those issues matches the <a href="../ag/index.html#config-rid"><code class="source">rid</code></a> configuration parameter); </p></li> <li><p> Issues not replicated by any replicator (that is, the replicator identifier for those issues is blank). The replicator will want to consider these issues as candidates for replication. </p></li> </ol> <p> Omit from this list: </p> <ol> <li><p> Issues replicated by a different replicator (that is, the replicator identifier for those issues differs from the <a href="../ag/index.html#config-rid"><code class="source">rid</code></a> configuration parameter). </p></li> <li><p> Issues known to be up to date with Perforce; either because they are unchanged since they were last replicated, or because they have only been changed by the replicator (<a href="#section-4.6">4.6</a> and <a href="#section-4.7">4.7</a>). </p></li> </ol> </li> <li><p> The empty list <code class="source">[ ]</code>. (This is for symmetry with the Perforce interface, which returns a list of changelists. Since changelists are not editable in the defect tracker, there's nothing that can be returned here, hence the empty list.) </p></li> <li><p> A marker. This must be some token that identifies what has been done on this poll. At the end of the replication cycle it will be passed to <a href="#defect_tracker.mark_changes_done"><code class="source">mark_changes_done()</code></a>. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration uses the record number of the last record in the <code class="source">TS_CHANGES</code> table that the replicator looked at as the marker indicating what it's done. See <a href="../../code/replicator/dt_teamtrack.py"><code class="source">dt_teamtrack.py</code></a> and the design [<a href="../../design/teamtrack-p4dti-schema/index.html#section-3.5">GDR 2000-09-04, 3.5</a>]. </p></blockquote></li> </ol> <p> This method must not record that the issues it returns have been considered for replication or replicated. The replicator can encounter an error during the course of replication that prevents it from making any progress (Perforce can go down, the defect tracker can go down, the replicator can crash). When the system comes back up, the replicator must re-consider these issues and possibly replicate them again. This helps keep the databases consistent (<a href="http://www.ravenbrook.com/project/p4dti/req/#req-1">requirement 1</a>) and is consistent with the design principle that the replicator must have no internal state. </p> <p> Recording that issues have been replicated must be left for the end of each replication cycle, when the marker (item 3 in the tuple) will be passed to <a href="#defect_tracker.mark_changes_done"><code class="source">mark_changes_done()</code></a>.</p> <h4> <a name="defect_tracker.init" id="defect_tracker.init"><code class="source">init(self)</code></a> </h4> <p> This method is called each time the replicator starts. </p> <p> The method must initialize the defect tracking database so that it is ready to start replication. The tables and fields in your schema extensions (see <a href="#section-4">section 4</a>) must be added if they are not yet present. </p> <h4> <a name="defect_tracker.issue" id="defect_tracker.issue"><code class="source">issue(self, issue_id)</code></a> </h4> <p> Return the defect tracking issue identified by the <code class="source">issue_id</code> argument, or None if there is no such issue. The returned issue (if any) must belong to your subclass of the <code class="source">dt_interface.defect_tracker_issue</code> class (see <a href="#section-7.2">section 7.2</a>). </p> <p> The <code class="source">issue_id</code> argument is a string identifying the issue (see <a href="#section-7.2.1">section 7.2.1</a>). </p> <h4> <a name="defect_tracker.mark_changes_done" id="defect_tracker.mark_changes_done"><code class="source">mark_changes_done(self, marker)</code></a> </h4> <p> This method is called at the end of each replication cycle, when all issues have been replicated. </p> <p> The <code class="source">marker</code> argument is the third item in the tuple returned by the <a href="#defect_tracker.changed_entities"><code class="source">changed_entities()</code></a> method at the start of the replication cycle. </p> <p> This method must now record that it has considered all changes up to the start of this replication cycle and replicated them successfully, so that at the next replication cycle it can ignore these changes and consider a new set of changes (see <a href="#section-4.6">section 4.6</a>). </p> <h4> <a name="defect_tracker.replicate_changelist" id="defect_tracker.replicate_changelist"><code class="source">replicate_changelist(self, change, client, date, description, status, user)</code></a> </h4> <p> Replicate a changelist to the defect tracker database (see <a href="#section-4.2">section 4.2</a>). </p> <p> The arguments specify the changelist; these arguments correspond to a subset of the fields in the changelist relation in the Perforce database (the names of the actual files changed, and their new revision numbers, are not replicated). </p> <ol> <li><p> The <code class="source">change</code> argument, an integer, is the change number. </p></li> <li><p> The <code class="source">client</code> argument, a string, is the client on which the change was last modified. </p></li> <li><p> The <code class="source">date</code> argument is the date and time at which the change was last modified. It has been converted by the date translator (see <a href="#section-7.5.1">section 7.5.1</a>). </p></li> <li><p> The <code class="source">description</code> argument is the change comment. It has been converted by the text translator (see <a href="#section-7.5.3">section 7.5.3</a>). </p></li> <li><p> The <code class="source">status</code> argument is a string. It is <code class="source">"pending"</code> if the changelist is pending, <code class="source">"submitted"</code> otherwise. </p></li> <li><p> The <code class="source">user</code> argument is the user who last modified the changelist. It has been converted by the user translator (see <a href="#section-7.5.4">section 7.5.4</a>). </p></li> </ol> <p> This method must return 1 if the changelist was new or changed, or 0 if it was unchanged. </p> <h3><a name="section-7.2" id="section-7.2">7.2. The <code class="source">dt_interface.defect_tracker_issue</code> class</a></h3> <p> A subclass of <code class="source">dt_interface.defect_tracker_issue</code> implements the replicator's interface to the issues in a defect tracker. </p> <blockquote><p> <i>Examples.</i> The <a href="../../code/replicator/dt_teamtrack.py"><code class="source">dt_teamtrack.py</code></a> module defines a class <code class="source">teamtrack_case</code> (issues are called "cases" in TeamTrack). The <a href="../../code/replicator/dt_bugzilla.py"><code class="source">dt_bugzilla.py</code></a> module defines a class <code class="source">bugzilla_bug</code> (issues are called "bugs" in Bugzilla). </p></blockquote> <h4><a name="section-7.2.1" id="section-7.2.1">7.2.1. Issue identifiers</a></h4> <p> The replicator needs a unique identifier for each issue in the defect tracker. This must be a string, so that it can be stored in the <code class="source">P4DTI-issue-id</code> field in the Perforce jobspec [<a href="../../design/replicator/index.html#section-4.2">GDR 2000-09-13, 4.2</a>]. The replicator gets the identifier from an issue's <a href="#defect_tracker_issue.id"><code class="source">id()</code></a> method. Later, it may pass the identifier to the defect tracker's <a href="#defect_tracker.issue"><code class="source">issue()</code></a> method. </p> <blockquote><p> <i>Example.</i> TeamTrack uniquely identifies issues by their record number in the database. So in <code class="source">dt_teamtrack</code> module, the issue identifier is the string conversion of the record number. </p></blockquote> <h4><a name="section-7.2.2" id="section-7.2.2">7.2.2. Issues are dictionaries</a></h4> <p> The replicator considers an issue to consist of a collection of named fields, with a value for each field. Instances of the <code class="source">defect_tracker_issue</code> subclass must support at least the <a href="#defect_tracker_issue.__getitem__"><code class="source">__getitem__()</code></a> method, so that the replicator can get the value for a field in an issue using the expression <code class="source">issue["fieldname"]</code>. </p> <p> You may want to implement the whole of the Python dictionary interface for your own use, but the replicator only uses <code class="source">__getitem__()</code>.</p> <p> A subclass of <code class="source">dt_interface.defect_tracker_issue</code> must define the following methods: </p> <h4> <a name="defect_tracker_issue.__getitem__" id="defect_tracker_issue.__getitem__"><code class="source">__getitem__(self, field)</code></a> </h4> <p> Return the value of the field named by the <code class="source">field</code> argument. Raise <code class="source">KeyError</code> if the issue has no such field. </p> <h4> <a name="defect_tracker_issue.__str__" id="defect_tracker_issue.__str__"><code class="source">__str__(self)</code></a> </h4> <p> Return a string describing the issue, suitable for presentation to a user or administrator in a report. Having several lines of the form "field name: value" should be fine. </p> <h4> <a name="defect_tracker_issue.add_fix" id="defect_tracker_issue.add_fix"><code class="source">add_fix(self, change, client, date, status, user)</code></a> </h4> <p> Add a fix to the issue (see <a href="#section-4.3">section 4.3</a>). </p> <p> The arguments specify the fix; these arguments correspond to the fields in the fix relation in the Perforce database. </p> <ol> <li><p> The <code class="source">change</code> argument, an integer, is the Perforce change number. </p></li> <li><p> The <code class="source">client</code> argument, a string, is the Perforce client name from which the fix was made. </p></li> <li><p> The <code class="source">date</code> argument is the date the fix was made. It has been converted by the date translator (see <a href="#section-7.5.1">section 7.5.1</a>). </p></li> <li><p> The <code class="source">status</code> argument, a string, is the effect of the fix. It is the status the job was changed to when the fix was made (or if the fix is to a pending changelist, then this is the status the job will be changed to when the changelist is submitted). </p> <p> The status is also known as the "effect" (for example, in the defect tracker's interface to fixes (see <a href="#section-9">section 9</a>)) because it gives the effect on the job when the fix is submitted. </p></li> <li><p> The <code class="source">user</code> argument is the user who made the fix. It has been converted by the user translator (see <a href="#section-7.5.4">section 7.5.4</a>). </p></li> </ol> <h4> <a name="defect_tracker_issue.corresponding_id" id="defect_tracker_issue.corresponding_id"><code class="source">corresponding_id(self)</code></a> </h4> <p> If this issue has been replicated, return the name of the Perforce job to which this issue is replicated. </p> <p> If this issue has not yet been replicated, return the name for the Perforce job to which this issue will be replicated. The returned value must be legal as the name of a Perforce job. You may want to use the result of the <a href="#defect_tracker_issue.corresponding_id"><code class="source">readable_name()</code></a> method if that is suitable. </p> <h4> <a name="defect_tracker_issue.filespecs" id="defect_tracker_issue.filespecs"><code class="source">filespecs(self)</code></a> </h4> <p> Return a list of the filespecs associated with this issue. Each item in the list belongs to your subclass of the <code class="source">defect_tracker_filespec</code> class (see <a href="#section-7.4">section 7.4</a>). </p> <h4> <a name="defect_tracker_issue.fixes" id="defect_tracker_issue.fixes"><code class="source">fixes(self)</code></a> </h4> <p> Return a list of the fixes for this issue. Each item in the list belongs to your subclass of the <code class="source">defect_tracker_fix</code> class (see <a href="#section-7.3">section 7.3</a>). </p> <h4> <a name="defect_tracker_issue.id" id="defect_tracker_issue.id"><code class="source">id(self)</code></a> </h4> <p> Return a string that can be used to uniquely identify this issue among all the issues in the defect tracker and to fetch it in future (see <a href="#section-7.2.1">section 7.2.1</a>). </p> <h4> <a name="defect_tracker_issue.readable_name" id="defect_tracker_issue.readable_name"><code class="source">readable_name(self)</code></a> </h4> <p> Return a string giving a human-readable name for the issue, as a string. This name is only used in logs and e-mail messages. </p> <h4> <a name="defect_tracker_issue.replicate_p" id="defect_tracker_issue.replicate_p"><code class="source">replicate_p(self)</code></a> </h4> <p> A function that decides, for an issue that is not yet replicated, whether it should be replicated by the running replicator. It must return 1 if the issue should be replicated, or 0 if it should not. </p> <p> The replicator calls this method for each issue returned by the defect tracker's <a href="#defect_tracker.changed_entities"><code class="source">changed_entities()</code></a> method which is not replicated: that is, its <a href="#defect_tracker_issue.rid"><code class="source">rid()</code></a> method returns the empty string. </p> <p> If this method returns 1, then the next thing the replicator will do is to call the issue's <a href="#defect_tracker_issue.setup_for_replication"><code class="source">setup_for_replication()</code></a> method. </p> <p> The configuration parameter <a href="../ag/index.html#config-replicate_p"><code class="source">replicate_p</code></a> can be used by the <abbr>P4DTI</abbr> administrator to set this method, for example if the organization only wants to replicate issues for a selected set of projects. </p> <h4> <a name="defect_tracker_issue.rid" id="defect_tracker_issue.rid"><code class="source">rid(self)</code></a> </h4> <p> Return the replicator identifier of the replicator that is in charge of replicating this issue, or the empty string if the issue is not being replicated. </p> <h4> <a name="defect_tracker_issue.setup_for_replication" id="defect_tracker_issue.setup_for_replication"><code class="source">setup_for_replication(self, jobname)</code></a> </h4> <p> Set up the issue for replication. That is, record that the issue is replicated by this replicator and record any other information in the database that is needed to replicate this issue. </p> <p> You must do at least these three steps: </p> <ol> <li><p> Record that the issue is replicated by this replicator, so that in the future its <a href="#defect_tracker_issue.rid"><code class="source">rid()</code></a> method returns the correct replicator identifier (this is the <code class="source">rid</code> parameter in the configuration passed to the defect tracker class when it was instantiated). </p></li> <li><p> Record the Perforce server identifier of the Perforce server it is replicated to (this is the <code class="source">sid</code> parameter in the configuration passed to the defect tracker class when it was instantiated). </p></li> <li><p> Record that the issue is replicated to the Perforce job named by the <code class="source">jobname</code> argument, so that in future its <a href="#defect_tracker_issue.corresponding_id"><code class="source">corresponding_id()</code></a> method returns <code class="source">jobname</code>. </p></li> </ol> <p> See <a href="#section-4.1">section 4.1</a>. </p> <h4> <a name="defect_tracker_issue.update" id="defect_tracker_issue.update"><code class="source">update(self, user, changes)</code></a> </h4> <p> Update the issue in the defect tracker's database. </p> <p> The <code class="source">user</code> argument is the user who made the change. It has been converted by the user translator (see <a href="#section-7.5.4">section 7.5.4</a>). </p> <p> The <code class="source">changes</code> argument is a dictionary of the changes that must be applied to the issue. The keys of the dictionary are the names of the fields that have changed; the values are the new values for those fields. Each value in the dictionary has been converted by the appropriate translator. If <code class="source">changes</code> is the empty dictionary, then do nothing. </p> <p> If the defect tracker supports transitions in a workflow, then this method should deduce the transition to apply (if any) based on the old and new values for the issue fields. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration attempts to find and apply a transition when the <code class="source">STATE</code> field changes. It looks at all the available transitions for the issue and selects the transition that would cause result in the correct new state. </p></blockquote> <blockquote><p> <i>Example.</i> Bugzilla doesn't have transitions, so there's no need for the Bugzilla integration to deduce one. </p></blockquote> <p> This method must check that the proposed change to the issue is legal in the defect tracker. (The changed fields have been converted by their translators, so each is legal individually, but the defect tracker may be more stringent, for example it may require a field not to have a value when the issue is in a particular state.) It must also check that the user has permission to make the proposed change. It's best if you can call a function in the defect tracker's API to apply the defect tracker's own rules (this is likely to be robust and maintainable), but if there's no such function, then you must do your best to emulate the defect tracker's checks. </p> <p> If the issue can't be updated (for example, because the user doesn't have permission to make the change, or because no workflow transition can be discovered, or because the proposed change is illegal in some way) then this method must raise an error. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration calls the <code class="source">TSServer::Transition</code> method in the TeamShare API, which checks the issue for correctness and checks that the user has the correct privilege. All the integration needs to do is raise an error when the function rejects the transition. </p></blockquote> <blockquote><p> <i>Example.</i> Bugzilla has no API, so the Bugzilla integration must emulate Bugzilla's checking. The <a href="../../code/replicator/dt_bugzilla.py"><code class="filename">dt_bugzilla.py</code></a> module defines three checking methods: <code class="source">restrict_fields()</code>, <code class="source">enforce_invariants()</code>, and <code class="source">check_permissions()</code>. </p></blockquote> <h3><a name="section-7.3" id="section-7.3">7.3. The <code class="source">dt_interface.defect_tracker_fix</code> class</a></h3> <p> A subclass of <code class="source">dt_interface.defect_tracker_fix</code> implements the replicator's interface to a fix record in a defect tracker (see <a href="#section-4.3">section 4.3</a>). </p> <blockquote><p> <i>Examples.</i> The <a href="../../code/replicator/dt_teamtrack.py"><code class="source">dt_teamtrack.py</code></a> module defines a class <code class="source">teamtrack_fix</code>. The <a href="../../code/replicator/dt_bugzilla.py"><code class="source">dt_bugzilla.py</code></a> module defines a class <code class="source">bugzilla_fix</code>. </p></blockquote> <p> A subclass of <code class="source">dt_interface.defect_tracker_fix</code> must define the following methods: </p> <h4> <a name="defect_tracker_fix.change" id="defect_tracker_fix.change"><code class="source">change(self)</code></a> </h4> <p> Return the change number for the fix, an integer. </p> <h4> <a name="defect_tracker_fix.delete" id="defect_tracker_fix.delete"><code class="source">delete(self)</code></a> </h4> <p> Delete the fix in the defect tracker so that the change is no longer linked to the issue. </p> <h4> <a name="defect_tracker_fix.status" id="defect_tracker_fix.status"><code class="source">status(self)</code></a> </h4> <p> Return the status of the fix, a string. </p> <h4> <a name="defect_tracker_fix.update" id="defect_tracker_fix.update"><code class="source">update(self, change, client, date, status, user)</code></a> </h4> <p> Update this fix in the defect tracker so that has the given fields. If the fields are unchanged, do nothing. </p> <p> This method is called when someone makes a new fix between the change and issue of an existing fix (for example, the status used to be "open", but now is "closed"). Since there can be only one fix for a given change and issue, the replicator updates the fix rather than creating a new fix. </p> <p> The arguments specify the fix; these arguments correspond to the fields in the fix relation in the Perforce database. </p> <ol> <li><p> The <code class="source">change</code> argument, an integer, is the Perforce change number. This is never changed. </p></li> <li><p> The <code class="source">client</code> argument, a string, is the Perforce client name from which the fix was made. </p></li> <li><p> The <code class="source">date</code> argument is the date the fix was made. It has been converted by the date translator (see <a href="#section-7.5.1">section 7.5.1</a>). </p></li> <li><p> The <code class="source">status</code> argument, a string, is the effect of the fix. It is the status the job was changed to when the fix was made (or if the fix is to a pending changelist, then this is the status the job will be changed to when the changelist is submitted). </p> <p> The status is also known as the "effect" (for example, in the defect tracker's interface to fixes (see <a href="#section-9">section 9</a>)) because it gives the effect on the job when the fix is submitted. </p></li> <li><p> The <code class="source">user</code> argument is the user who made the fix. It has been converted by the user translator (see <a href="#section-7.5.4">section 7.5.4</a>). </p></li> </ol> <h3><a name="section-7.4" id="section-7.4">7.4. The <code class="source">dt_interface.defect_tracker_filespec</code> class</a></h3> <p> A subclass of <code class="source">dt_interface.defect_tracker_filespec</code> implements the replicator's interface to a filespec record in a defect tracker (see <a href="#section-4.4">section 4.4</a>). </p> <blockquote><p> <i>Examples.</i> The <a href="../../code/replicator/dt_teamtrack.py"><code class="source">dt_teamtrack.py</code></a> module defines a class <code class="source">teamtrack_filespec</code>. The <a href="../../code/replicator/dt_bugzilla.py"><code class="source">dt_bugzilla.py</code></a> module defines a class <code class="source">bugzilla_filespec</code>. </p></blockquote> <p> A subclass of <code class="source">dt_interface.defect_tracker_filespec</code> must define the following methods: </p> <h4> <a name="defect_tracker_filespec.delete" id="defect_tracker_filespec.delete"><code class="source">delete(self)</code></a> </h4> <p> Delete the filespec record so that the issue is no longer associated with the filespec. </p> <h4> <a name="defect_tracker_filespec.name" id="defect_tracker_filespec.name"><code class="source">name(self)</code></a> </h4> <p> Return the filespec, a string. </p> <h3><a name="section-7.5" id="section-7.5">7.5. The <code class="source">translator.translator</code> class</a></h3> <p> A subclass of <code class="source">translator.translator</code> translates values of a particular type between the defect tracker and Perforce. You should define a translator for each field type in the defect tracker that you want the <abbr>P4DTI</abbr> administrator to be able to replicate. You must define translators for dates (see <a href="#section-7.5.1">section 7.5.1</a>), multi-line text fields (see <a href="#section-7.5.3">section 7.5.3</a>), and users (see <a href="#section-7.5.4">section 7.5.4</a>). If your defect tracker has any concept of the state of an issue, then you must define a translator for states (see <a href="#section-7.5.2">section 7.5.2</a>). </p> <blockquote><p> <i>Example.</i> The TeamTrack integration defines, in addition to the three required translators, translators for: fields that cross-reference an auxiliary table like <code class="source">TS_PROJECTS</code>; elapsed time fields; selection fields; and the <code class="source">STATE</code> field. </p></blockquote> <p> The translator base class doesn't know anything about Perforce; all it knows is that it is translating between two defect trackers, called 0 and 1. In the <abbr>P4DTI</abbr>, defect tracker 1 is always Perforce, but we haven't limited the design of the translator class by requiring that it is. </p> <p> Each subclass of <code class="source">translator.translator</code> must define the following methods: </p> <h4> <a name="translator.translate_0_to_1" id="translator.translate_0_to_1"><code class="source">translate_0_to_1(self, value, dt0, dt1, issue0=None, issue1=None)</code></a> </h4> <p> Return <code class="source">value</code>, suitably translated from defect tracker 0 to defect tracker 1. If translation is not possible, raise an error. </p> <ol> <li><p> The <code class="source">value</code> argument is a value in a field in an issue in defect tracker 0. </p></li> <li><p> The <code class="source">dt0</code> argument is your defect tracker: an instance of your subclass of <a href="#section-7.1"><code class="source">dt_interface.defect_tracker</code></a>. </p></li> <li><p> The <code class="source">dt1</code> argument is Perforce (represented by an instance of a subclass of <a href="#section-7.1"><code class="source">dt_interface.defect_tracker</code></a>). </p></li> <li><p> The issue in your defect tracker from which the value comes, or <code class="source">None</code> if the value doesn't come from an issue. An instance of your subclass of <a href="#section-7.2"><code class="source">dt_interface.defect_tracker_issue</code></a>. </p></li> <li><p> The job in Perforce to which the value is going, or <code class="source">None</code> if the value isn't going to a job (represented by an instance of a subclass of <a href="#section-7.2"><code class="source">dt_interface.defect_tracker_issue</code></a>). </p></li> </ol> <p> This method takes defect trackers as arguments because it may need to query the defect tracker to carry out the translation. </p> <blockquote><p> <i>Example.</i> In the TeamTrack integration, the single select translator needs to read the <code class="source">TS_SELECTIONS</code> table to discover the available selections. To do this it calls the private method <code class="source">read_selections()</code> in <code class="source">dt0</code>. </p></blockquote> <p> This method takes issues as arguments because some translators need to know about the whole issue in order to carry out the translation. </p> <blockquote><p> <i>Example.</i> In the TeamTrack integration the state translator needs to know the project to which the issue belongs (because different projects may have different states with the same name which correspond to the same Perforce state). </p></blockquote> <p> Many translators can ignore the <code class="source">dt0</code> and <code class="source">dt1</code> arguments; most can ignore the <code class="source">issue0</code> and <code class="source">issue1</code> arguments. </p> <h4> <a name="translator.translate_1_to_0" id="translator.translate_1_to_0"><code class="source">translate_1_to_0(self, value, dt0, dt1, issue0=None, issue1=None)</code></a> </h4> <p> Return <code class="source">value</code>, suitably translated from defect tracker 1 to defect tracker 0. If translation is not possible, raise an error. </p> <ol> <li><p> The <code class="source">value</code> argument is a value in a field in an issue in defect tracker 1 (that is, in a job in Perforce). </p></li> <li><p> The <code class="source">dt0</code> argument is your defect tracker: an instance of your subclass of <a href="#section-7.1"><code class="source">dt_interface.defect_tracker</code></a>. </p></li> <li><p> The <code class="source">dt1</code> argument is Perforce (represnted by an instance of a subclass of <a href="#section-7.1"><code class="source">dt_interface.defect_tracker</code></a>). </p></li> <li><p> The issue in your defect tracker 0 to which the value is going, or <code class="source">None</code> if the value isn't going to an issue. An instance of your subclass of <a href="#section-7.2"><code class="source">dt_interface.defect_tracker_issue</code></a>. </p></li> <li><p> The job in Perforce from which the value comes, or <code class="source">None</code> if the value doesn't come from a job (represented by an instance of a subclass of <a href="#section-7.2"><code class="source">dt_interface.defect_tracker_issue</code></a>). </p></li> </ol> <h4><a name="section-7.5.1" id="section-7.5.1">7.5.1. Date translator</a></h4> <p> You must define a date translator class, a subclass of <a href="#section-7.5"><code class="source">translator.translator</code></a>, to translate dates between your defect tracker and Perforce. </p> <p> When translating to Perforce: </p> <ol> <li><p> An empty or null date field must be translated to the empty string. </p></li> <li><p> Any other date must be translated to a string looking like <code class="source">"2000/12/31 23:59:59"</code> (you can do this by calling <code class="source">time.strftime()</code> with <code class="source">"%Y/%m/%d %H:%M:%S"</code> as the first argument). </p></li> </ol> <p> When translating from Perforce: </p> <ol> <li><p> The empty string must be translated to an empty or null date field. </p></li> <li><p> A string in the format <code class="source">"2000/12/31 23:59:59"</code> specifies the calendar date. (This form is used by changelists and jobs.) </p></li> <li><p> A string consisting only of digits specifies the number of seconds since 1970-01-01 00:00:00. (This form is used by fixes.) </p></li> </ol> <blockquote><p> <i>Example.</i> TeamTrack specifies all dates as seconds since 1970-01-01 00:00:00, so the TeamTrack integration uses <code class="source">time.strftime()</code> to convert from TeamTrack to Perforce, and either <code class="source">time.mktime()</code> or simply <code class="source">int()</code> to convert from Perforce to TeamTrack.</p></blockquote> <h4><a name="section-7.5.2" id="section-7.5.2">7.5.2. State translator</a></h4> <p> If your defect tracker has a concept of states for issues, then you must define a state translator class, a subclass of <a href="#section-7.5"><code class="source">translator.translator</code></a>. </p> <p> The state field in Perforce should be a "select" field (see <a href="#section-8.4">section 8.4</a>) so the values for this field should be legal selections in Perforce. This means no whitespace, hashes, double quotes, semicolons or slashes. Since the defect tracker probably allows these character to appear in state names, you must convert them somehow. </p> <p> We have provided a translator to do this conversion: it is the <code class="source">keyword_translator</code> class in the <a href="../../code/replicator/translator.py"><code class="filename">translator.py</code></a> module. </p> <p> You shouldn't just use the keyword translator as your state translator, since all it does is to convert strings. You should develop a translator that checks that applies the keyword translator, checks that the converted state is legal and raises and error if it is not. </p> <h4><a name="section-7.5.3" id="section-7.5.3">7.5.3. Text translator</a></h4> <p> You must define a text translator class, a subclass of <a href="#section-7.5"><code class="source">translator.translator</code></a>, to translate multi-line text fields between your defect tracker and Perforce. </p> <p> This translator must translate line endings (if needed). Perforce uses newline (<code class="source">"\n"</code>) as the line ending; values always end in a newline (unless the field is empty); values never end in more than one newline. </p> <blockquote><p> <i>Example.</i> TeamTrack uses a carriage return plus a newline (<code class="source">\r\n</code>) as its line ending, and there need not be a final newline. </p></blockquote> <h4><a name="section-7.5.4" id="section-7.5.4">7.5.4. User translator</a></h4> <p> You must define a user translator class, a subclass of <code class="source">translator.user_translator</code>, to translate users between your defect tracker and Perforce. </p> <p> It is important not to assume that userids are the same in Perforce and the defect tracker, because an organization may have different policies for assigning userids in the two systems, or there may be legacy users from a previous policy. Therefore, the translator should translate between users based on their e-mail addresses. </p> <p> When translating from the defect tracker to Perforce: </p> <ol> <li><p> Map the defect tracker user to a Perforce user with the same e-mail address, if there is one. </p></li> <li><p> Otherwise, map the defect tracker user to the Perforce user with the same userid, if there is one. </p></li> <li><p> Otherwise, return the defect tracker userid unchanged (assuming it is valid syntactically as a Perforce userid; if it isn't, you should apply the keyword translator (see <a href="#section-7.5.2">section 7.5.2</a>) to it). </p></li> </ol> <p> When translating from Perforce to the defect tracker: </p> <ol> <li><p> Map the Perforce user to a defect tracker user with the same e-mail address, if there is one. </p></li> <li><p> Otherwise, map the Perforce user to the defect tracker user with the same userid, if there is one. </p></li> <li><p> Otherwise, if translating the user in a changelist or fix, map the Perforce user to some dummy defect tracker user (see <a href="#section-4.8">section 4.8</a>). You can tell that you're translating a changelist or fix rather than an issue because the <code class="source">issue0</code> argument to the <a href="#translator.translate_1_to_0"><code class="source">translate_1_to_0()</code></a> method is <code class="source">None</code>. </p></li> <li><p> Otherwise, you're translating a user field in an issue and you can't find a match either by e-mail address or by name. Raise an error. </p></li> </ol> <p> Each subclass of <code class="source">translator.user_translator</code> must define the following method: </p> <h4> <a name="user_translator.unmatched_users" id="user_translator.unmatched_users"><code class="source">unmatched_users(self)</code></a> </h4> <p> This method should examine all the users in the defect tracker and Perforce and return a report on the users in each system that have no corresponding userid in the other. </p> <p> It must return a tuple of four elements: </p> <ol> <li><p> A dictionary of users in the defect tracker that have no corresponding userid in Perforce. The keys of the dictionary are strings naming the defect tracker userids; the values of the dictionary are the e-mail addresses of the defect tracker users. </p></li> <li><p> A dictionary of users in Perforce that have no corresponding userid in the defect tracker. The keys of the dictionary are the Perforce userids; the values of the dictionary are the e-mail addresses of the Perforce users. </p></li> <li> <p> A comment (a string) about the users in the first dictionary explaining how they will be treated by this user translator. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration says, "These TeamTrack users will appear as themselves in Perforce even though there is no such Perforce user." </p></blockquote> </li> <li> <p> A comment (a string) about the users in the second dictionary explaining how they will be treated by this user translator. </p> <blockquote><p> <i>Example.</i> The TeamTrack integration says, "These Perforce users will appear in TeamTrack as the user (None). It will not be possible to assign issues to these users." </p></blockquote> </li> </ol> <p> This method is called each time the replicator is started. The results are used to compose an e-mail to the <abbr>P4DTI</abbr> administrator reporting on unmatched users. </p> <h2><a name="section-8" id="section-8">8. Configuration</a></h2> <p> This section describes how to configure the <abbr>P4DTI</abbr> to work with your extension. To understand how the configuration works, see [<a href="../../design/replicator/index.html#section-5">GDR 2000-09-13, 5</a>]. </p> <h3><a name="section-8.1" id="section-8.1">8.1. The configuration generator</a></h3> <p> You must write a configuration generator for your defect tracker. This must be a module called <code class="filename">config_<i>defect_tracker</i>.py</code>, where <i>defect_tracker</i> is the name you chose for your defect tracker (see <a href="#section-3">section 3</a>), converted to lower case. </p> <p> It must provide the following function: </p> <h4><a id="configure_dt.configuration" name="configure_dt.configuration"><code class="source">configuration(config)</code></a></h4> <p> The <code class="source">config</code> argument is a module whose members are the configuration parameters specified by the <abbr>P4DTI</abbr> administrator in <a href="../../code/replicator/config.py"><code class="filename">config.py</code></a>. </p> <p> It must check all the user configuration parameters that are specific to your defect tracker. </p> <p> It must return a tuple of two elements: </p> <ol> <li><p> A Perforce jobspec (see <a href="#section-8.4">section 8.4</a>), or <code class="source">None</code> if the Perforce jobspec should not be changed. </p></li> <li><p> A revised configuration module, which should include all the configuration parameters in the <code class="source">config</code> argument, plus the configuration parameters required by your defect tracker module, by the replicator (see <a href="#section-8.3">section 8.3</a>), and by the Perforce interface (see <a href="#section-8.2">section 8.2</a>). </p></li> </ol> <h3><a name="section-8.2" id="section-8.2">8.2. Perforce interface configuration</a></h3> <p> The revised configuration module must include the following parameter for the Perforce interface. (This is in addition to the parameters <a href="../ag/index.html#config-p4_client_executable"><code class="source">p4_client_executable</code></a>, <a href="../ag/index.html#config-p4_password"><code class="source">p4_password</code></a>, <a href="../ag/index.html#config-p4_port"><code class="source">p4_port</code></a>, and <a href="../ag/index.html#config-p4_user"><code class="source">p4_user</code></a> which came from the user configuration.) </p> <h4><a name="config.logger" id="config.logger"><code class="source">logger</code></a></h4> <p> This is a logger object (see <a href="#section-5.3">section 5.3</a>) to which log messages will be written. It must log to <a href="../ag/index.html#config-log_file"><code class="source">log_file</code></a> if that is specified, to standard output, and to any appropriate system logging facility. It must respect the <a href="../ag/index.html#config-log_level"><code class="source">log_level</code></a>. </p> <h3><a name="section-8.3" id="section-8.3">8.3. Replicator configuration</a></h3> <p> The revised configuration module must include the following parameters for the replicator. (These are in addition to the <a href="../ag/index.html#config-administrator_address"><code class="source">administrator_address</code></a>, <a href="../ag/index.html#config-p4_user"><code class="source">p4_user</code></a>, <a href="../ag/index.html#config-poll_period"><code class="source">poll_period</code></a>, <a href="../ag/index.html#config-replicate_p"><code class="source">replicate_p</code></a>, <a href="../ag/index.html#config-replicator_address"><code class="source">replicator_address</code></a>, <a href="../ag/index.html#config-rid"><code class="source">rid</code></a>, and <a href="../ag/index.html#config-smtp_server"><code class="source">smtp_server</code></a> parameters which came from the user configuration.) </p> <h4><a name="config.date_translator" id="config.date_translator"><code class="source">date_translator</code></a></h4> <p> A date translator instance (see <a href="#section-7.5.1">section 7.5.1</a>). </p> <h4><a name="config.field_map" id="config.field_map"><code class="source">field_map</code></a></h4> <p> A description of how fields map from the defect tracker to Perforce and back again. It is a list of tuples, one for each field to be replicated. Each tuple has three elements: </p> <ol> <li><p> The name of the field in the defect tracker. </p></li> <li><p> The name of the field in Perforce. </p></li> <li><p> A translator instance (see <a href="#section-7.5">section 7.5</a>) that can be used to translate between values in the two fields. </p></li> </ol> <p> The field map must match the defect tracker database and the Perforce jobspec (see <a href="#section-8.4">section 8.4</a>). </p> <h4><a name="config.job_owner_field" id="config.job_owner_field"><code class="source">job_owner_field</code></a></h4> <p> The name of the field in the Perforce jobspec which contains the owner of the job. </p> <blockquote><p> <i>Example.</i> In the TeamTrack integration, this is <code class="source">"Owner"</code>. In the Bugzilla integration, this is <code class="source">"Assigned_To"</code>. </p></blockquote> <h4><a name="config.job_status_field" id="config.job_status_field"><code class="source">job_status_field</code></a></h4> <p> The name of the field in the Perforce jobspec which contains the status of the job. </p> <blockquote><p> <i>Example.</i> In the TeamTrack integration, this is <code class="source">"State"</code>. In the Bugzilla integration, this is <code class="source">"Status"</code>. </p></blockquote> <h4><code class="source">logger</code></h4> <p> This must be the same as the <a href="#config.logger">logger object for the Perforce interface</a>. </p> <h4><a name="config.text_translator" id="config.text_translator"><code class="source">text_translator</code></a></h4> <p> A text translator instance (see <a href="#section-7.5.3">section 7.5.3</a>). </p> <h4><a name="config.user_translator" id="config.user_translator"><code class="source">user_translator</code></a></h4> <p> A user translator instance (see <a href="#section-7.5.4">section 7.5.4</a>). </p> <h3><a name="section-8.4" id="section-8.4">8.4. Perforce jobspecs</a></h3> <p> The configuration generator must build a Perforce jobspec that matches the <a href="#config.field_map"><code class="source">field_map</code></a> configuration parameter that it generates. </p> <p> The jobspec must be a Perforce dictionary suitable for passing to <code class="command">p4 -G jobspec -i</code>. Such a jobspec has keys called "Fields<i>n</i>", "Values<i>n</i>" and "Presets<i>n</i>". The numbers in the keys are arbitrary, but they must correspond (so the presets for "Field12" must be in "Presets12"). </p> <p> Perforce requires that five fields be present: </p> <ol> <li><p> The jobname ("101 Job word 32 required"). This must not be replicated. </p></li> <li><p> The status of the job (field 102). This should be replicated from the defect tracker. It should be a "select" field. Its values should include all the statuses for issues in the defect tracker, converted using the state translator (see <a href="#section-7.5.2">section 7.5.2</a>). </p></li> <li><p> The owner of the job (field 103). This should be replicated from the defect tracker. It should be a "word" field. </p></li> <li><p> The date the job was last modified ("104 Date date 20 always"). This must not be replicated. </p></li> <li><p> The title (field 104). This should be replicated from the defect tracker. It should be a "line" or "text" field. </p></li> </ol> <blockquote><p> <i>Example.</i> In the TeamTrack integration, Perforce's five required fields appear in the jobspec like this: </p></blockquote> <blockquote><code class="source"> 'Fields0': '101 Job word 32 required, <br /> 'Fields1': '102 State select 32 required', <br /> 'Presets1': '_new', <br /> 'Values1': '_new/assigned/closed/verified/deferred', <br /> 'Fields2': '103 Owner word 32 required', <br /> 'Presets2': '$user', <br /> 'Fields3': '104 Date date 20 always', <br /> 'Presets3': '$now, <br /> 'Fields4': '105 Title line 80 required, <br /> 'Presets4': '$blank, <br /> </code></blockquote> <p> The replicator requires that four fields be present [<a href="../../design/replicator/index.html#section-4">GDR 2000-09-13, 4</a>]. These should appear in the jobspec as follows: </p> <blockquote><code class="source"> 'Fields<i>n</i>': '191 P4DTI-filespecs text 0 optional', <br /> 'Fields<i>n+1</i>': '192 P4DTI-rid word 32 required', <br /> 'Presets<i>n+1</i>': 'None', <br /> 'Fields<i>n+2</i>': '193 P4DTI-issue-id word 32 required', <br /> 'Presets<i>n+2</i>': 'None', <br /> 'Fields<i>n+3</i>': '194 P4DTI-user word 32 always', <br /> 'Presets<i>n+3</i>': '$user', <br /> </code></blockquote> <p> These fields have high numbers so that they appear at the bottom of the jobspec where people don't have to look at them. </p> <p> The remainder of the jobspec should be filled in with the fields that the <abbr>P4DTI</abbr> administrator has specified for replication in the <a href="../ag/index.html#config-replicated_fields"><code class="source">replicated_fields</code></a> configuration parameter. Make sure that the values for "select" fields are legal in Perforce (see <a href="#section-7.5.2">section 7.5.2</a>). </p> <p> Finally, the jobspec should have a "Comments" key. The value for this is a string, each line starting with <code class="source">"# "</code>. You should include comments for as many fields as you can, taking information from the defect tracker's database if that's possible. </p> <blockquote> <p> <i>Example.</i> The comment for the jobspec in the TeamTrack integration may look like this: </p> <code class="source"> # DO NOT CREATE NEW JOBS IN PERFORCE. USE TEAMTRACK. <br /> # <br /> # Job: The job name. <br /> # State: The state of the job in the TeamTrack workflow. <br /> # Owner: The person responsible for taking action. <br /> # Date: The date this job was last modified. <br /> # Title: A short description of the problem from the user's point of view. <br /> # P4DTI-rid: P4DTI replicator identifier. Do not edit! <br /> # P4DTI-issue-id: TeamTrack issue database identifier. Do not edit! <br /> # P4DTI-user: Last user to edit this job. You can't edit this! <br /> </code> <p> Here, the comments for the State, Owner and Title fields have been taken from the field descriptions in the TeamTrack database. The other comments have been supplied by the TeamTrack configuration generator. </p> <p> For more information about jobspecs, see Chapter 5, "Customizing Perforce: Job Specifications", in the Perforce user's guide [<a href="http://www.perforce.com/perforce/doc.002/manuals/p4sag/05_jobspec.html#1037386" id="ref-Perforce-2000-12-22b" name="ref-Perforce-2000-12-22b">Perforce 2000-12-22b, 5</a>]. </p> </blockquote> <h3><a name="section-8.5" id="section-8.5">8.5. Adapting the configuration module</a></h3> <p> If your defect tracker module (see <a href="#section-7">section 7</a>) requires the <abbr>P4DTI</abbr> administrator to specify any configuration parameters (such as the hostname on which the defect tracker runs, or the user to connect to the database as), then you must adapt the <a href="../../code/replicator/config.py"><code class="filename">config.py</code></a> module, as follows. </p> <ol> <li><p> Add a new <code class="source"># dt_name = "<i>Defect_Tracker</i>"</code> line near the start of section 2, to indicate that your defect tracker integration is available. </p></li> <li><p> Add a new subsection to section 3, starting <code class="source">elif dt_name == "<i>Defect_Tracker</i>":</code>. This should contain default values for the configuration parameters required only by your integration. </p></li> <li><p> Add a history entry to Appendix B explaining what you've done. </p></li> </ol> <h3><a name="section-8.6" id="section-8.6">8.6. Making your own configurations</a></h3> <p><strong>Warning: The configuration methods in this section are not supported by Perforce or TeamShare.</strong></p> <p> This section describes techniques you can use if you want to adapt a supported integration to do something that's not supported. Here are some of the things that are possible by making your own configuration. </p> <ol> <li> Connecting a defect tracker to multiple Perforce servers. </li> <li> Working with an existing Perforce jobs database and an existing Perforce jobspec (see <a href="#section-8.6.3">section 8.6.3</a>). </li> <li> Fine control over which defect tracker issues are replicated to Perforce. </li> <li> Fine control over which issue fields are replicated, and their names (see <a href="#section-8.6.2">section 8.6.2</a>). </li> <li> Arbitrary translation of issue data between the defect tracker and Perforce (see <a href="#section-8.6.3">section 8.6.3</a>). </li> <li> Queries based on combined defect tracker and Perforce data. </li> <li> Working with locally customized versions of Bugzilla. </li> <li> Redirection or fine control of logging. </li> <li> Arbitrary mappings between Perforce and defect tracker user names. </li> <li> Alternative policies for handling conflicts between defect tracker and Perforce changes to issues. </li> </ol> <h4><a name="section-8.6.1" id="section-8.6.1">8.6.1. Steps to making your own configuration</a></h4> <p> Here's are the steps you need to follow to make your own configuration: </p> <ol> <li><p> Choose a name for your configuration: <i>my_configuration</i>, say. </p></li> <li> <p> Edit <code class="filename">config.py</code>, adding the line</p> <blockquote><code class="source">configure_name = '<i>my_configuration</i>'</code></blockquote> </li> <li><p> Make a new module <code class="filename">configure_<i>my_configuration</i>.py</code>. </p></li> <li><p> Make your new module into a configuration generator (see <a href="#section-8.1">section 8.1</a>). See below for some examples. </p></li> </ol> <p> The best approach to making a configuration generator is to use an existing one and modify its output. That way, you benefit from improvements and corrections to the configuration generator in future releases of the <abbr>P4DTI</abbr>. </p> <h4><a name="section-8.6.2" id="section-8.6.2">8.6.2. Example: change the field names in Perforce</a></h4> <p> Suppose that you are using the TeamTrack integration, but you have tools that work on Perforce jobs that have assumptions about the names of fields in the jobspec. You want the fields in the Perforce jobspec to be called Status, User and Description, not State, Owner and Title. If the Description field is replicated from TeamTrack, you want it to be called Long_Description in Perforce. </p> <p> In this case, your configuration generator should look like this: </p> <blockquote><code class="source"> import configure_teamtrack <br /> import re <br /> import string <br /> <br /> convert = { <br /> 'State': 'Status', <br /> 'Owner': 'User', <br /> 'Title': 'Description', <br /> 'Description': 'Long_Description', <br /> } <br /> <br /> def configuration(config): <br /> jobspec, revised_config = configure_teamtrack.configuration(config) <br /> <br /> # Convert field names in the jobspec. <br /> for k,v in jobspec.items(): <br /> match = re.match("^([0-9]+) (.*) ([a-z]+) ([0-9]+) ([a-z]+)$", v) <br /> if match and convert.has_key(match.groups(2)): <br /> groups = match.groups() <br /> groups[1] = convert[groups[1]] <br /> jobspec[k] = string.join(groups, ' ') <br /> <br /> # Convert field names in the field_map. <br /> for field_map in revised_config.field_map: <br /> if convert.has_key(field_map[0]): <br /> field_map[0] = convert[field_map[0]] <br /> <br /> return jobspec, revised_config </code> </blockquote> <h4><a name="section-8.6.3" id="section-8.6.3">8.6.3. Example: use existing Perforce jobspec</a></h4> <p> Suppose that you want to use the TeamTrack integration, but you have many existing tools and documents that refer to your current Perforce jobspec, so don't want the <abbr>P4DTI</abbr> to change your Perforce jobspec. </p> <p> First, you must add the fields required by the <abbr>P4DTI</abbr> [<a href="../../design/replicator/index.html#section-4">GDR 2000-09-13, 4</a>] to your Perforce jobspec by hand. Then your configuration generator should look like this: </p> <blockquote><code class="source"> import configure_teamtrack <br /> import message <br /> import translator <br /> <br /> product = "My configuration" <br /> error = "My configuration error." <br /> catalog = { <br /> 1: (message.ERR, "Unknown TeamTrack state '%s'."), <br /> 2: (message.ERR, "Unknown Perforce status '%s'."), <br /> } <br /> factory = <a href="#section-5.2">message.catalog_factory</a>(catalog, product) <br /> <br /> # A map from state in TeamTrack to status in Perforce. <br /> state_pairs = [ <br /> (1, 'open'), # Assigned <br /> (2, 'closed'), # Resolved <br /> (5, 'suspended'), # Deferred <br /> ] <br /> <br /> class my_state_translator(<a href="#section-7.5">translator.translator</a>): <br /> def translate_0_to_1(self, value, dt0, dt1, issue0 = None, issue1 = None): <br /> assert isinstance(value, types.IntType) <br /> for (t,p) in state_pairs: <br /> if value == t: <br /> return p <br /> raise error, factory.new(1, value) <br /> <br /> def translate_1_to_0(self, value, dt0, dt1, issue0 = None, issue1 = None): <br /> assert isinstance(value, types.StringType) <br /> for (t,p) in state_pairs: <br /> if value == p: <br /> return t <br /> raise error, factory.new(2, value) <br /> <br /> def configuration(config): <br /> _, revised_config = configure_teamtrack.configuration(config) <br /> <br /> # Make a field_map that works with my existing jobspec. <br /> revised_config.field_map = [ <br /> ('STATE', 'Status', my_state_translator()), <br /> ('OWNER', 'User', revised_config.<a href="#config.user_translator">user_translator</a>), <br /> ('TITLE', 'Description', <a href="#section-7.5">translator.translator</a>()), <br /> ('DESCRIPTION', 'User_Impact', revised_config.<a href="#config.text_translator">text_translator</a>), <br /> ] <br /> <br /> # Return None as the jobspec so the Perforce jobspec won't be changed. <br /> return None, revised_config <br /> </code> </blockquote> <p> Note the use of coding conventions in this example: message catalogs (see <a href="#section-5.2">section 5.2</a>) and raising exceptions when a value can't be translated (see <a href="#section-5.4">section 5.4</a>). </p> <p> <b>Warning.</b> If you leave your Perforce jobspec unchanged, you must check that it is compatible with the <abbr>P4DTI</abbr>. The reason for this is that the replicator uses <code class="command">p4 -G job -o <i>jobname</i></code> to get a job from Perforce; this command applies more stringent checking than <code class="command">p4 job -o <i>jobname</i></code>. In particular, you must: </p> <ol> <li> <p> Check that the "Presets" for each select field is valid for that field (that is, it appears as one of the "Values" for that field). </p> <p> Some organizations set up a jobspec with a field like this: </p> <blockquote><code class="source"> Fields: 120 Severity select 20 required <br /> Values: Severity critical/essential/optional <br /> Presets: Severity setme </code></blockquote> <p> Their intention is that since "setme" is not a legal value for the Severity field, the person submitting the job must give it a value; they can't just ignore it and leave it with the default value. </p> <p> However, this won't work with the <abbr>P4DTI</abbr>, because the command <code class="command">p4 -G job -o</code> won't even give you a blank job form; instead it gives you an error message. </p> </li> </ol> <h2><a name="section-9" id="section-9">9. Testing</a></h2> <p> See the <a href="../../test/index.html"><code class="filename">test/</code></a> directory of the integration kit for test procedures, sample data and automated test cases that can be used to test the integration. </p> <p> You should adapt the existing tests so that they test your integration. </p> <p> [ This section needs a lot more work. Sources of information: the <a href="../../test/index.html">index to the test directory</a>; the <a href="../../test/test_p4dti.py"><code class="filename">test_p4dti.py</code></a> test script; other test cases in that directory; our testing procedures; the Python unit testing framework [<a href="http://pyunit.sourceforge.net/">Purcell 2001-02-12</a>]; the <a href="http://www.ravenbrook.com/project/p4dti/issue/?display=Job:Test:Title&view=status%3dclosed+product%3dp4dti">closed P4DTI issues</a> (these are suitable for regression testing). GDR 2001-03-23] </p> <h2><a name="section-10" id="section-10">10. Providing a defect tracker interface to Perforce relations</a></h2> <p> The defect tracker should display, for each issue that is replicated, a description of the Perforce server to which the issue is replicated. Use the configuration parameter <a href="../ag/index.html#config-p4_server_description"><code class="source">p4_server_description</code></a> which you should have stored in a table in the defect tracker (see <a href="#section-4.5">section 4.5</a>). </p> <p> The defect tracker should display on each issue description page a table of fixes for that issue (if there are any). The table should look like the table below. </p> <table border="1" cellspacing="0" cellpadding="5"> <tr valign="top" align="left"> <th>Change</th> <th>Effect</th> <th>Date</th> <th>User</th> <th>Description</th> </tr> <tr valign="top"> <td><a href="http://info.ravenbrook.com/infosys/cgi/perfbrowse.cgi?@describe+5493">5493</a></td> <td>open</td> <td>2000-12-05</td> <td>GDR</td> <td>Added replicator method mail_concerning_job() for e-mailing people about a job.</td> </tr> <tr valign="top"> <td><a href="http://info.ravenbrook.com/infosys/cgi/perfbrowse.cgi?@describe+5524">5524</a></td> <td>open</td> <td>2000-12-06</td> <td>GDR</td> <td>Fixed the replicator's user_email_address method so that it really returns None when there is no such user.</td> </tr> <tr valign="top"> <td><a href="http://info.ravenbrook.com/infosys/cgi/perfbrowse.cgi?@describe+5541">5541</a></td> <td>open</td> <td>2000-12-06</td> <td>GDR</td> <td>If the owner of a job and the person who last changed it are the same, include them only once in any e-mail sent by the replicator about that job.</td> </tr> <tr valign="top"> <td><a href="http://info.ravenbrook.com/infosys/cgi/perfbrowse.cgi?@describe+5634">5634</a> (pending)</td> <td>closed</td> <td>2000-12-07</td> <td>GDR</td> <td>Merging back to master sources.</td> </tr> </table> <p> Points to note about this table: </p> <ol> <li><p> Pending changelists are distinguished from submitted changelists. This is important because the effect of a pending changelist does not happen until the changelist is submitted. So in the above table the status of the job is still "open" but it is understood that when changelist 5634 is submitted it will become "closed". </p></li> <li><p> The user and date are for the change (not for the fix). Knowing when the change was made and by whom is much more important than knowing when the change was linked with the job. </p></li> <li><p> The user is the defect tracker user who corresponds to the Perforce user who made the change. </p></li> <li><p> The change number is a link to the URL given by the <a href="../ag/index.html#config-changelist_url"><code class="source">changelist_url</code></a> configuration parameter, with the change number inserted. This configuration parameter is defined in the <cite>Administrator's Guide</cite> as being suitable for passing to <code class="source">sprintf()</code> as the format string: it must have one <code>%d</code> format specified (for which the change number will be substituted) and it may have any number of doubled percent signs <code>%%</code> (which must become single percent signs in the resulting URL) [<a href="../ag/index.html#section-5.1">RB 2000-08-10a, 5.1</a>]. </p></li> <li><p> All the fixes for an issue will be being replicating by the same replicator and from the same Perforce server as the issue itself. So when building this table you only need to select records with the same replicator identifier and Perforce server identifier as the issue. </p></li> </ol> <h2><a name="section-11" id="section-11">11. Adapting the manuals</a></h2> <p> When adding material relating to your defect tracker to the manuals, surround each section with the HTML tags <code class="source"><div class="<i>defect_tracker</i>"></code> and <code class="source"></div></code>. This makes the material for a particular defect tracker easy to find, extract and check, to meet <a href="http://www.ravenbrook.com/project/p4dti/req/#req-32">requirement 32</a>. </p> <p> You must adapt the <cite>Perforce Defect Tracking Integration Administrator's Guide</cite> [<a href="../ag/index.html">RB 2000-08-10a</a>] to describe your integration, as described in the list below. </p> <ol> <li><p> Add a new subsection to section 3, specifying the software and procedural prerequisites for using your defect tracker with the <abbr>P4DTI</abbr>. </p></li> <li><p> If your integration requires a new installation procedure, or installs on a new platform, update section 4. </p></li> <li><p> Add your new configuration parameters to section 5.1. </p></li> <li><p> Add a new subsection to section 5, explaining how to configure your defect tracker for the <abbr>P4DTI</abbr>. </p></li> <li><p> Add a new item to the list in section 10, explaining how to uninstall your integration and return your defect tracker to its original state. </p></li> <li><p> Add the error messages that your code can produce to section 11.2 (include the product, message identifier and check digit just like the other errors messages in that section). </p></li> <li><p> Add likely error messages from systems with which your code interacts to section 11.3 (for example, errors from the defect tracker, or from your database interface). </p></li> <li><p> Add references to documentation for your defect tracker, and any other supporting materials that you referred to, to appendix A. </p></li> <li><p> Add a history entry to appendix B explaining what you've done. </p></li> </ol> <p> If you provided an interface from your defect tracker to the Perforce fixes relation (see <a href="#section-10">section 10</a>), then you must adapt the <cite>Perforce Defect Tracking Integration User's Guide</cite> [<a href="../ug/index.html">RB 2000-08-10b</a>] to describe your integration, as follows: </p> <ol> <li><p> Add a paragraph to section 10.3 explaining how to access Perforce fixes from the defect tracker. </p></li> </ol> <h2><a name="section-12" id="section-12">12. Making your work available to the community</a></h2> <h3><a name="section-12.1" id="section-12.1">12.1. Reporting defects</a></h3> <p> Defects in the <abbr>P4DTI</abbr> Kit include (but aren't limited to): </p> <ol> <li><p> An essential piece of information can't be found in this manual or in the design documents it refers to. </p></li> <li><p> Inconsistencies between this manual, the design documents it refers to, and the sources they document. </p></li> <li><p> Defects in the <abbr>P4DTI</abbr> sources or in the test cases. </p></li> </ol> <p> Please report any defects you find to <a href="http://www.perforce.com/perforce/support.html">Peforce support</a>, so that they can fixed and the product improved. </p> <p> Please provide the following information with your defect report: </p> <ol> <li><p> The release of the <abbr>P4DTI</abbr> Kit you are using (look in the <code class="filename">readme.txt</code> that came with the <abbr>P4DTI</abbr> Kit to identify the release). </p></li> <li><p> The name and release of the defect tracker you are integrating with. </p></li> <li><p> If you're reporting a defect in documentation: </p> <ol> <li><p> What you're trying to do. </p></li> <li><p> The information you need. </p></li> <li><p> Where you expected to find it. </p></li> <li><p> Where else you looked for it. </p></li> </ol></li> <li><p> If you're reporting a defect in the code: </p> <ol> <li><p> What you did immediately prior to the defect's occurrence. </p></li> <li><p> What you think should have happened. </p></li> <li><p> What actually happened. </p></li> <li><p> The Perforce release you are using. </p></li> <li><p> Any source code you've added or modified, including your <code class="filename">config.py</code> file. </p></li> <li><p> A section of the <abbr>P4DTI</abbr> log that includes the error that you're reporting and some context around that error. </p></li> <li><p> Copies of any related e-mail messages generated by the <abbr>P4DTI</abbr>. </p></li> </ol></li> </ol> <h3><a name="section-12.2" id="section-12.2">12.2. Making a contribution</a></h3> <p> Please send your contributions (fixes, adaptions and extensions) to Perforce support [<a href="http://www.perforce.com/perforce/support.html">Support</a>]. Please include the following: </p> <ol> <li><p> A description of your contribution: what it is designed to achieve; which files you've changed; which files you've added. </p></li> <li><p> The release of the <abbr>P4DTI</abbr> Kit you have been developing against (look in the <code class="filename">readme.txt</code> that came with the <abbr>P4DTI</abbr> Kit to identify the release). </p></li> <li><p> The complete <abbr>P4DTI</abbr> Kit, including your modifications and additions. Make a tarball or a ZIP archive of the whole <abbr>P4DTI</abbr> Kit directory. (Please do this even if you've only changed a couple of files. This allows us to add your contribution to Perforce and use <code class="command">p4 diff2</code> to see exactly what changes you've made.) </p></li> <li><p> What you are prepared for us to do with your contribution. Are you willing for us to make it available for distribution from Perforce or Ravenbrook's web site? Are you willing for us to incorporate it into the <abbr>P4DTI</abbr> and maintain and support it? Have you made it available under an open source license? </p></li> </ol> <h2><a name="section-A" id="section-A">A. References</a></h2> <table> <tr valign="top"> <td>[<a name="ref-GDR-2000-05-03" id="ref-GDR-2000-05-03" href="http://www.ravenbrook.com/project/p4dti/doc/2000-05-03/reqs-and-use-cases/">GDR 2000-05-03</a>]</td> <td> "Requirements and Use Cases for Perforce/Defect Tracking Integration"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-05-03. </td> </tr> <tr valign="top"> <td> [<a href="http://www.ravenbrook.com/project/p4dti/req/" id="ref-GDR-2000-05-24" name="ref-GDR-2000-05-24">GDR 2000-05-24</a>] </td> <td> "Perforce Defect Tracking Integration Project Requirements"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-05-24. </td> </tr> <tr valign="top"> <td>[<a name="ref-GDR-2000-05-30" id="ref-GDR-2000-05-30" href="/project/p4dti/doc/2000-05-30/arch-analysis/">GDR 2000-05-30</a>]</td> <td> "Analysis of architectures for defect tracking integration"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-05-30. </td> </tr> <tr valign="top"> <td>[<a id="GDR-2000-08-08" name="GDR-2000-08-08" href="../../design/python-teamtrack-interface/index.html">GDR 2000-08-08</a>]</td> <td> "Python interface to TeamTrack: design"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-08-08. </td> </tr> <tr valign="top"> <td>[<a id="GDR-2000-09-04" name="GDR-2000-09-04" href="../../design/teamtrack-p4dti-schema/index.html">GDR 2000-09-04</a>]</td> <td> "TeamTrack database schema extensions for integration with Perforce"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-09-04. </td> </tr> <tr valign="top"> <td>[<a id="GDR-2000-09-13" name="GDR-2000-09-13" href="../../design/replicator/index.html">GDR 2000-09-13</a>]</td> <td> "Replicator design"; <a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>; <a href="http://www.ravenbrook.com/">Ravenbrook</a>; 2000-09-13. </td> </tr> <tr valign="top"> <td valign="top">[<a name="ref-Lutz-1996" id="ref-Lutz-1996">Lutz 1996</a>]</td> <td valign="top"> "Programming Python"; Mark Lutz; O'Reilly; 1996-10; ISBN 1-56592-197-6. </td> </tr> <tr valign="top"> <td>[<a id="NB-2000-11-14a" name="NB-2000-11-14a" href="../../design/bugzilla-schema/index.html">NB 2000-11-14a</a>]</td> <td> "Bugzilla database schema"; <a href="mailto:nb@ravenbrook.com">Nick Barnes</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-11-14. </td> </tr> <tr valign="top"> <td>[<a id="NB-2000-11-14b" name="NB-2000-11-14b" href="../../design/bugzilla-p4dti-schema/index.html">NB 2000-11-14b</a>]</td> <td> "Bugzilla database schema extensions for integration with Perforce"; <a href="mailto:nb@ravenbrook.com">Nicholas Barnes</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-11-14. </td> </tr> <tr valign="top"> <td>[<a id="NB-2000-11-14c" name="NB-2000-11-14c" href="../../design/python-bugzilla-interface/index.html">NB 2000-11-14c</a>]</td> <td> "Python interface to Bugzilla: design"; <a href="mailto:nb@ravenbrook.com">Nicholas Barnes</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-11-14. </td> </tr> <tr valign="top"> <td>[<a href="http://www.perforce.com/perforce/doc.002/manuals/p4guide/" id="ref-Perforce-2000-12-22a" name="ref-Perforce-2000-12-22a">Perforce 2000-12-22a</a>]</td> <td> "Perforce 2000.2 P4 Command Line User's Guide"; <a href="http://www.perforce.com/">Perforce Software</a>; 2000-12-22; <<a href="http://www.perforce.com/perforce/doc.002/manuals/p4guide/">http://www.perforce.com/ perforce/doc.002/ manuals/p4guide/</a>>, <<a href="ftp://ftp.perforce.com/pub/perforce/r00.2/doc/manuals/p4guide/p4guide.pdf">ftp://ftp.perforce.com/ /pub/perforce/r00.2/doc/ manuals/p4guide/p4guide.pdf</a>>. </td> </tr> <tr valign="top"> <td>[<a href="http://www.perforce.com/perforce/doc.002/manuals/p4sag/" id="ref-Perforce-2000-12-22b" name="ref-Perforce-2000-12-22b">Perforce 2000-12-22b</a>]</td> <td> "Perforce 2000.2 System Administrator's Guide"; <a href="http://www.perforce.com/">Perforce Software</a>; 2000-12-22; <<a href="http://www.perforce.com/perforce/doc.002/manuals/p4sag/">http://www.perforce.com/ perforce/doc.002/ manuals/p4sag/</a>>, <<a href="ftp://ftp.perforce.com/pub/perforce/r00.2/doc/manuals/p4sag/p4sag.pdf">ftp://ftp.perforce.com/ /pub/perforce/ r00.2/doc/ manuals/p4sag/p4sag.pdf</a>>. </td> </tr> <tr valign="top"> <td> [<a href="http://pyunit.sourceforge.net/" id="ref-Purcell-2001-02-12" name="ref-Purcell-2001-02-12">Purcell 2001-02-12</a>] </td> <td> "PyUnit — a unit testing framework for Python"; <a href="mailto:stephen_purcell@yahoo.com">Steve Purcell</a>; 2001-02-12. </td> </tr> <tr valign="top"> <td> [<a href="../ag/index.html" id="ref-RB-2000-08-10a" name="ref-RB-2000-08-10a">RB 2000-08-10a</a>] </td> <td> "Perforce Defect Tracking Integration Administrator's Guide"; <a href="mailto:rb@ravenbrook.com">Richard Brooksby</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-08-10. </td> </tr> <tr valign="top"> <td> [<a href="../ug/index.html" id="ref-RB-2000-08-10b" name="ref-RB-2000-08-10b">RB 2000-08-10b</a>] </td> <td> "Perforce Defect Tracking Integration User's Guide"; <a href="mailto:rb@ravenbrook.com">Richard Brooksby</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-08-10. </td> </tr> <tr valign="top"> <td>[<a id="RB-2000-08-10c" name="RB-2000-08-10c" href="../../design/architecture/index.html">RB 2000-08-10c</a>]</td> <td> "Perforce Defect Tracking Integration Architecture"; <a href="mailto:rb@ravenbrook.com">Richard Brooksby</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2000-08-10. </td> </tr> <tr valign="top"> <td>[<a id="RB-2001-03-07" name="RB-2001-03-07" href="http://www.ravenbrook.com/project/p4dti/procedure/contribution/">RB 2001-03-07</a>]</td> <td> "P4DTI Project Contributions Procedure"; <a href="mailto:rb@ravenbrook.com">Richard Brooksby</a>; <a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>; 2001-03-07. </td> </tr> <tr valign="top"> <td>[<a id="van-Rossum-2000-10-16" name="van-Rossum-2000-10-16" href="http://www.python.org/doc/current/tut/tut.html">van Rossum 2000-10-16</a>]</td> <td> "Python Tutorial"; <a href="mailto:python-docs@python.org">Guido van Rossum</a>; 2000-10-16. </td> </tr> </table> <h2><a name="section-B" id="section-B">B. Document History</a></h2> <table> <tr valign="top"> <td>2000-10-16</td> <td><a href="mailto:rb@ravenbrook.com">RB</a></td> <td>Created placeholder after meeting with <a href="mailto:lmb@ravenbrook.com">LMB</a>.</td> </tr> <tr valign="top"> <td>2000-12-10</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Drafted sections 3 and 4.</td> </tr> <tr valign="top"> <td>2000-12-11</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Drafted sections 2, 5, and 8 and outlined sections 6 and 7.</td> </tr> <tr valign="top"> <td>2000-12-31</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>The table of fixes in section 8 now distinguishes pending from submitted changes.</td> </tr> <tr valign="top"> <td>2001-01-02</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Added section 7.1 (configuration architecture), figure 1, section 7.5 (customized configuration). Moved text from appendix D of the Administrator's Guide to section 7.5.</td> </tr> <tr valign="top"> <td>2001-02-04</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Updated definition of <code class="source">defect_tracker.all_issues</code> method.</td> </tr> <tr valign="top"> <td>2001-02-23</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Added <code class="source">corresponding_id</code> method; revised definition of <code class="source">readable_name</code> method.</td> </tr> <tr valign="top"> <td> 2001-03-02 </td> <td> <a href="mailto:rb@ravenbrook.com">RB</a> </td> <td> Transferred copyright to Perforce under their license. </td> </tr> <tr valign="top"> <td>2001-03-13</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Deleted the recording of conflicts and the need for manual conflict resolution. Conflict resolution is always immediate.</td> </tr> <tr valign="top"> <td>2001-03-20</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Added overviews of requirements, architecture and design. Included replicator block diagram. Improved links.</td> </tr> <tr valign="top"> <td>2001-03-21</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Wrote specifications of the defect tracker and translator classes. Described messages, catalogs, logging and errors. Moved logging and errors to new section 5 because it's important; renumbered remaining sections; moved configuration adaption to new section 8.6.</td> </tr> <tr valign="top"> <td>2001-03-22</td> <td><a href="mailto:gdr@ravenbrook.com">GDR</a></td> <td>Specified the configuration generator; explained how to make your own configuration (with example code); explained how to adapt the manuals. Gave warning about incompatible jobspecs. Added prerequisites and where to get help. Added section on testing. Added section on code layout. Explained how to report defects and submit contributions.</td> </tr> <tr valign="top"> <td> 2001-03-29 </td> <td> <a href="mailto:rb@ravenbrook.com">RB</a> </td> <td> Changed internal and external cross-references to conform with our other documents. Tidied up references to requirements. Removed claims about assigning copyright. Removed section promising to do particular things with contributions. Updated references to Perforce manuals to (latest) release 2000.2. Validated and fixed some broken links. Sorted references section. </td> </tr> </table> <hr /> <p> <small>This document is copyright © 2001 Perforce Software, Inc. All rights reserved.</small> </p> <p> <small>Redistribution and use of this document in any form, with or without modification, is permitted provided that redistributions of this document retain the above copyright notice, this condition and the following disclaimer.</small> </p> <p> <small> <strong> This document is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this document, even if advised of the possibility of such damage. </strong> </small> </p> <div align="center"> <p><code class="perforce-keyword">$Id: //info.ravenbrook.com/project/p4dti/release/1.1.1/ig/index.html#1 $</code></p> </div> </body> </html>
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#1 | 5093 | Hari Krishna Dara |
Populating perforce branch. I will be adding p4admin files to it. |
||
//guest/perforce_software/p4dti/release/1.1.1/ig/index.html | |||||
#1 | 585 | richard_brooksby | Adding P4DTI release 1.1.1 and updating the P4DTI index page. |