Figure 1. The replication architecture
Perforce Defect Tracking Integration Project
This manual is the Perforce Defect Tracking Integration 1.0 Administrator's Guide. It explains how to install, configure, maintain, and administer the Perforce Defect Tracking Integration (P4DTI).
This document is intended for P4DTI administrators. Ordinary users of the defect tracker or Perforce should read the Perforce Defect Tracking Integration User's Guide. (For ideas on how to train your users on the P4DTI, see Section 8, "Training and documentation".)
This guide does not describe the basics of using the P4DTI, Perforce, or the defect tracker. Read the Perforce Defect Tracking Integration User's Guide to understand the P4DTI from a user's perspective.
To install and run the P4DTI, you must:
The P4DTI works by taking over the job tracking system of Perforce and making the defect tracker's records appear as Perforce jobs. Perforce users can work with jobs more or less as described in the Perforce manuals, and their changes are reflected in the defect tracker. For more information on how Perforce handles jobs, see the Perforce Command Line User's Guide.
Perforce has a mechanism for linking jobs to changelists, to enable you to record the work done for a particular reason. The P4DTI makes these links appear in the defect tracker, making it easy to see what was done or is currently being done to resolve a defect.
The P4DTI replicator is a process that copies data between a defect tracker and a Perforce server to keep each one up to date with changes made in the other. This approach allows developers to do their routine defect resolution work entirely from their Perforce client, without using the defect tracker's interface. It also allows developers to relate their changes to defect tracking issues.
Figure 1 shows how the replicator communicates with the defect tracking server and the Perforce server.
The replicator maintains a one-to-one relationship between issues in the defect tracker's database and jobs in the Perforce repository. (An issue is a unit of work that the defect tracker tracks; some examples are bugs, change requests, and enhancement requests.) In other words, each issue has a corresponding job, and vice versa. The replicator keeps the contents of a configurable set of fields in the defect tracker's issues the same as the contents of the corresponding Perforce job, so that editing one edits the other.
The replicator also copies Perforce's links between jobs and changelists (called "fixes") to the defect tracker's database, and makes them visible in the defect tracker's user interface. Replication of links from Perforce to the defect tracker makes it possible to track, record, and check a number of things; in particular, it makes it possible to track and record the changes made for each issue, and find out why a change was made in terms of issues.
The replicator polls the defect tracking server and the Perforce server at regular intervals to get a list of recent changes, and attempts to propagate these changes to the other system. If a defect tracker issue is changed at the same time as the corresponding Perforce job, the replicator sends an e-mail with the overwritten Perforce job data to the following people:
Most defect trackers have an idea of workflow--a set of rules that control who can do what to which issues. The replicator enforces the defect tracker's workflow by rejecting changes to jobs in Perforce that are illegal in the defect tracker. When it comes across such a change, it undoes the change and sends an e-mail message to the user.
The defect tracker manages the defect tracker records (and therefore the job contents), while Perforce manages the changelists. Neither side controls the "fixes" relationship--the links between jobs and changelists.
Figure 1 shows how the replicator connects to the Perforce and defect tracker servers.
Figure 1. The replication architecture
To administer the P4DTI, you must have the following experience:
Before installing the P4DTI, you must obtain and install the following software:
Before installing the P4DTI, you must do the following:
Before installing the P4DTI, you must obtain and install the following software:
Before installing the P4DTI, you must do the following:
Before installing the P4DTI, you must obtain and install the following software:
syslog module. An RPM of Python 1.5.2 for RedHat Linux 6.2 is available from <http://www.ravenbrook.com/project/p4dti/import/1999-04-13/python-1.5.2/python-1.5.2-13.i386.rpm>. An RPM of Python 1.5.2 for RedHat Linux 7.0 is available from <http://www.ravenbrook.com/project/p4dti/import/1999-04-13/python-1.5.2/python-1.5.2-27.i386.rpm>. Before installing the P4DTI, you must do the following:
Note: You might want to practice installing and configuring the P4DTI using a test Perforce repository and a test defect tracking database before you try it with your real data. A copy of your real Perforce repository would be ideal; for instructions on how to make a copy of your repository, see the Perforce System Administrator's Guide.
The P4DTI can be installed on any machine that can communicate with the defect tracker's server and the Perforce server. To keep administration simple and reduce network traffic, install and run the P4DTI on the same machine as the defect tracker's server. The rest of this manual assumes that you do this.
For instructions on how to upgrade from an earlier version of the P4DTI, see the release notes.
The P4DTI is distributed as a self-extracting executable called p4dti-DT-RELEASE.exe (where DT is the defect tracker, such as "teamtrack", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, run this executable on the machine where the defect tracker server is installed. The installer unpacks the P4DTI into C:\Program Files\P4DTI\ by default.
The P4DTI is distributed as an RPM called p4dti-DT-RELEASE-1.i386.rpm where DT is the defect tracker, such as "bugzilla", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, run the following command as root on the defect tracker server machine:
rpm -i p4dti-DT-RELEASE-1.i386.rpm This will install the P4DTI files into /opt/p4dti and a startup script in the /etc/rc.d/init.d directory.
If you prefer not to use RPMs, you can follow the procedure in section 4.4, "Solaris installation".
The P4DTI is distributed as a gzipped tar file called p4dti-DT-RELEASE.tar.gz (where DT is the defect tracker, such as "bugzilla", and RELEASE is the release number, such as "1.0.2").
To install the P4DTI, unpack this tar file on the defect tracker server machine, using the following command:
gunzip -c p4dti-DT-RELEASE.tar.gz | tar xvf -You must determine where to put the files. You can put the files wherever you want.
Work through the subsections in the order in which they appear. Do not attempt to run the P4DTI until you have reached the end of this section, or you might end up with a non-working installation.
To configure the P4DTI with Perforce and your defect tracker, you must:
To configure the P4DTI, you edit definitions of Python variables in the file config.py in the installation directory. Edit these definitions according to the notes below. All variables in the file must have a value.
rid Description: The replicator identifier.
Example: "replicator0"
Must be 32 characters or less, start with a letter or underscore, and consist only of letters, numbers, and underscores.
The replicator identifier is used to distinguish between replicators when multiple replicators are being used to replicate issues from a defect tracker to different Perforce servers. If you have only one replicator, it doesn't matter what you use for the replicator identifier; "replicator0" is a good choice since it allows you to add more replicators later.
If you change the replicator identifier then your currently replicated defect tracker issues will stop being replicated. The replicator will believe they are being handled by another replicator.
sid Description: The Perforce server identifier.
Example: "perforce0"
Must be 32 characters or less, start with a letter or underscore, and consist only of letters, numbers and underscores. You might want to use the hostname of your Perforce server, if it is stable.
administrator_address Description: The e-mail address of the P4DTI administrator.
Example: "p4dti-admin@company.domain"
The replicator sends error reports to this address. If this is
None, then the replicator will never send e-mail.
changelist_url Description: A format string used to build a URL for change descriptions. Specify None if there is no URL for change descriptions.
Example: "http://info.company.domain:8080/%d?ac=10"
The string is a format string valid for passing to sprintf(); it must have one %d format specifier, for which the change number is substituted. (Note that because it gets passed to sprintf(), you must specify other percent signs twice.) Defect trackers that support this feature list the changelists that fix each issue, and make a link from each changelist to this URL, with the change number substituted.
If you are using perfbrowse, then a valid format string looks like "http://info.company.domain/cgi/perfbrowse.cgi?@describe+%d". If you are using P4Web, then a valid format string looks like "http://info.company.domain:8080/%d?ac=10"
closed_state Description: The defect tracker state that maps to the "closed" state in Perforce. Specify None if you want the ordinary state mapping rules to apply.
Example: "Resolved" in TeamTrack; "RESOLVED" in Bugzilla.
Mapping the defect tracker state that developers use most often to the "closed" state in Perforce makes using the P4DTI easier for the developers, because the Perforce user interfaces make it easier to fix a job to "closed" than any other state. If you are using TeamTrack and your workflow already has a state called "Closed", then that state must map to "closed" in Perforce; set this variable to None. The "CLOSED" state in Bugzilla maps to "bugzilla_closed" in Perforce.
dbms_host Description: Bugzilla only. The host on which the Bugzilla MySQL server is running.
Example: "localhost"
Set this value to "localhost" if the P4DTI and the Bugzilla MySQL server run on the same machine.
dbms_database Description: Bugzilla only. The name of the MySQL database in which Bugzilla stores its data.
Example: "bugs"
Normally set to "bugs" during Bugzilla installation (see the Bugzilla README file). Change this setting only if you have set up Bugzilla differently.
dbms_port Description: Bugzilla only. The port number on which the Bugzilla MySQL server listens on the database host (dbms_host).
Example: 3306
MySQL normally listens on port 3306. Change this setting only if you have set up MySQL differently. Note that this parameter is expressed as a number, not as a string.
dbms_user Description: Bugzilla only. The user name that the replicator uses to log in to MySQL to use the Bugzilla database.
Example: "bugs"
Bugzilla normally logs in to MySQL as user "bugs" (see the Bugzilla README file). Change this setting only if you have configured Bugzilla differently, or if you want to set up the replicator to log in as a different user.
dbms_password Description: Bugzilla only. The password that the replicator uses to log in to MySQL to use the Bugzilla database.
Example: ""
Bugzilla normally logs in with no password (see the Bugzilla README file). Change this setting if you have configured Bugzilla differently, or you want to set up the replicator to log in as a different user and use a password.
bugzilla_directory Description: Bugzilla only. The directory in which Bugzilla is installed, or None if you don't want e-mail processed.
Example: "/home/httpd/html/bugzilla"
Bugzilla sends e-mail to its users when it notices that a bug has been changed. If the P4DTI is running on the Bugzilla server, it is able to use Bugzilla's processmail script to promptly send e-mail in the same way. This configuration parameter allows the P4DTI to locate processmail. Set it to None if the P4DTI is not running on the Bugzilla server or if you don't want the P4DTI to send these e-mail messages.
dt_name Description: The name of the defect tracking system you're integrating with. Either "TeamTrack" or "Bugzilla".
Example: "TeamTrack"
Make sure that this variable is set to the appropriate value for your defect tracker.
verbose Description: Specifies whether verbose logging is enabled.
Example: 0
If verbose logging is required, set this parameter to 1; to enable normal logging, set it to 0.
log_file Description: The name of the replicator's log file. If log messages should not be sent to a file, specify None.
Example: "C:\\Program Files\\P4DTI\\p4dti.log"
The replicator generates log messages to record its actions. These log messages are sent to all of the following locations:
p4_client_executable Description: The location of the Perforce client executable.
Example: "C:\\Program Files\\Perforce\\p4.exe"
This setting doesn't need to be an absolute path name if the directory is on the replicator user's path. On Windows this setting might be "C:\\Program Files\\Perforce\\p4.exe". On UNIX it might be just "p4".
p4_password Description: The password the replicator uses to log in to the Perforce server. If there is no password, specify "" (empty quotes).
Example: ""
For information about how the replicator logs in to Perforce, see Section 5.2.
p4_port Description: The address and port of the Perforce server with which the replicator communicates.
Example: "perforce.company.domain:1666"
p4_server_description Description: A description of the Perforce server. This might be used by the defect tracker to show which Perforce server an issue is replicated to.
Example: "Hardware development group Perforce server"
p4_user Description: The userid that the replicator uses to log in to the Perforce server.
Example: "p4dti-replicator0"
For information about how the replicator logs in to Perforce, see Section 5.2. If you want to add more replicators later, incorporate the replicator identifier (rid) into this userid.
poll_period Description: The period of time between the end of one poll of the servers and the start of the next, in seconds.
Example: 10
replicate_p Description: A function that selects which issues to start replicating. Normally, the P4DTI replicates all issues created or modified after the start_date, but you can modify this function to further restrict the issues. Some Python programming is required.
Example for TeamTrack that restricts replication to issues belong to the project whose ID is 6:
def replicate_p(self):
return self['PROJECTID'] == 6
Example for Bugzilla that restricts replication to unresolved issues in the "nosebag" product:
def replicate_p(self):
return self.bug['product'] == 'nosebag' and self.bug['resolution'] == ''
Note that once an issue starts being replicated it remains replicated, even if is no longer matches the criteria.
replicated_fields (for TeamTrack) Description: A list of the database names of TeamTrack fields that are replicated in Perforce. The fields STATE, OWNER, and TITLE are always replicated, so omit those fields when setting this variable.
Example: ["DESCRIPTION", "PRIORITY", "SEVERITY"]
For advice on which fields to replicate, and how to find out their database names, see Section 5.1.1.
replicated_fields (for Bugzilla) Description: A list of the names of Bugzilla fields that are replicated in Perforce. The fields "bug_status", "short_desc", "assigned_to" and "resolution" are always replicated, so omit those fields when setting this variable.
Example: ["longdesc", "priority", "bug_severity", "product"]
For advice on which fields to replicate, see Section 5.1.1.
replicator_address Description: The e-mail address from which the replicator sends e-mail. This address is used in the "From" field of e-mail that the replicator sends.
Example: "p4dti-replicator0@company.domain"
To make it easier for users to get assistance, make this address an alias for the administrator e-mail address (administrator_address). If you are using Bugzilla, this e-mail address is also used for the replicator's Bugzilla account; see Section 5.4.2.
smtp_server Description: The address of the SMTP server that the replicator uses to send e-mail.
Example: "smtp.company.domain"
If this is None, then the replicator will never send e-mail.
start_date Description: The starting point in time for replication.
Example: "2001-02-10 00:00:00"
Issues modified after
this date will be replicated; issues unchanged after this date will be
ignored. Must be a string in the form "YYYY-MM-DD
HH:MM:SS".
teamtrack_password Description: TeamTrack only. The password that the replicator uses to log into TeamTrack. If there is no password, specify "" (empty quotes).
Example: ""
See Section 5.3.2.
teamtrack_server Description: TeamTrack only. The TeamTrack server hostname and (optionally) port with which the replicator communicates.
Example: "teamtrack.company.domain:80"
(Note that "localhost" won't work, even
if the TeamTrack eerver is on the local host.)
teamtrack_user Description: TeamTrack only. The user name that the replicator uses to log into TeamTrack.
Example: "P4DTI-replicator0"
See Section 5.3.2.
Here's some advice on which fields to replicate:
If you're using TeamTrack's sample database, you might want to replicate the following fields:
To find out the database name of a TeamTrack field, follow these steps:
If you're using Bugzilla, you might want to replicate the following fields:
If you're using Bugzilla, the replicator rejects the following types of changes from within Perforce:
The following table lists the field names for Bugzilla 2.10. If you have modified Bugzilla, your field names may differ. To display the set of Bugzilla field names, type mysqlshow bugs bugs at a shell prompt.
Table 2. Bugzilla field names
| Field name | Name on Bugzilla form | Replication policy |
|---|---|---|
| bug_id | Bug # | always, read only |
| bug_status | Status | always, read/write |
| assigned_to | Assigned To | always, read/write, user |
| short_desc | Summary | always, read/write |
| resolution | Resolution | always, read/write |
| bug_file_loc | URL | read/write |
| bug_severity | Severity | read/write |
| op_sys | OS | read/write |
| priority | Priority | read/write |
| rep_platform | Platform | read/write |
| reporter | Reporter | read/write, user |
| qa_contact | - | read/write, user |
| status_whiteboard | Status Whiteboard | read/write |
| longdesc | Description | append only |
| groupset | - | read only |
| creation_ts | Opened | read only |
| delta_ts | - | read only |
| product | Product | read only |
| version | Version | read only |
| component | Component | read only |
| target_milestone | Target Milestone | read only |
| votes | Votes | read only |
| keywords | Keywords | read only |
| lastdiffed | - | read only |
| everconfirmed | - | read only |
The following fields are displayed on the Bugzilla bug form but are kept in separate database tables and cannot be replicated:
If you need to change the list of replicated fields after you've started using the P4DTI, see Section 9, "Maintaining the P4DTI".
To configure Perforce, you must:
Create a user in Perforce for the replicator; for instructions, see the Perforce System Administrator's Guide. The replicator user must have the following properties:
p4_user) that you specified in the P4DTI configuration. replicator_address). If you're using the Perforce protections, make the replicator a super user so that it can set the jobspec. For instructions, see the Perforce System Administrator's Guide. You'll need to add a line like
super user p4dti-replicator0 * //...to the protections list.
For information on getting a license from Perforce Software for this extra user, see Section 3.2, "Perforce prerequisites".
You can use the P4DTI in combination with a Perforce trigger to enforce extra workflow restrictions. For example, if your organization assigns priorities to issues, you can prevent changes being made to areas of the repository unless they resolve at least one defect of priority 3 or higher.
The P4DTI comes with an example trigger script that you can adapt for your needs, installed as example_trigger.py in the default installation directory.
To enforce workflow restrictions, follow these steps:
replicated_fields) configuration parameter. You must delete all jobs from your Perforce installation. The P4DTI takes over the jobs subsystem of Perforce and rewrites the Perforce jobspec.
For instructions, see the Perforce Command Line User's Guide.
If you already use Perforce jobs and have significant tools that depend on your jobspec, the configuration options described in Section 5.1, "P4DTI configuration", might not be flexible enough to support your requirements. However, you might be able to write your own configuration and use your own jobspec. To write your own configuration, you must understand the P4DTI configuration architecture and be fluent in the Python programming language. See the Perforce Defect Tracking Integration Integrator's Guide for details of how to configure the P4DTI and guidance on developing your own configuration. Note that neither Perforce nor the manufacturer of your defect tracker can support a configuration that you write yourself.
To configure TeamTrack, you must:
You need to add a TeamTrack value to the Windows Registry to tell TeamTrack that the P4DTI is present. To do this, double-click the p4dti.reg file that comes with the P4DTI (it's installed in c:\program files\p4dti\p4dti.reg by default).
You need to create a TeamTrack user for the replicator. This user corresponds to the replicator TeamTrack userid (teamtrack_user) parameter you set in Section 5.1, "P4DTI configuration".
To create a TeamTrack user for the replicator, follow these steps:
teamtrack_user) parameter). For information on getting a license from TeamShare for this extra user, see Section 3.3, "TeamTrack prerequisites".
Figure 3. New user: General tab
Figure 4. New user: Privileges tab
The replicator uses TeamTrack issue field descriptions as the source for the Perforce job field descriptions. These job field descriptions appear in comments in every job form (if you're using the Perforce command line) and as tooltips for the fields in the job editing dialog (if you're using P4Win, the Perforce Windows GUI).
TeamTrack leaves field descriptions blank when you create a database, so you must provide descriptions of fields that your developers edit. For example, you might describe the TITLE field as "A one-sentence statement of the problem from the user's perspective", and the DESCRIPTION field as "A detailed description of the problem from the user's perspective, including how to reproduce it."
To enter field descriptions, follow these steps:
To configure Bugzilla, you must:
You need to make some minor modifications to the Bugzilla code so that users can see Perforce information on Bugzilla bug forms. These modifications are distributed as a patchfile for version 2.10 of Bugzilla.
If you have modified Bugzilla at your site, you might still be able to apply the patch successfully. Changes to the database schema, the permissions rules, or the workflow rules are likely to cause the P4DTI to malfunction. You might need to modify the P4DTI if you have changed these parts of Bugzilla.
The patch changes the following Bugzilla files:
bug_form.pl, which adds a Perforce section to the bug form. defparams.pl, which adds a parameter to control whether or not the Perforce section appears. These changes are small and self-contained. If your changes do not affect these two files or only affect them in minor ways, the patch should operate correctly. If the patch program fails because of your Bugzilla modifications, it might still be possible to introduce the changes by hand. If you cannot apply the patch, the replicator might still work, but the Bugzilla bug form will not show Perforce-specific information (for example, changelists that are linked to the bug by a "fix"). The operation of the replicator itself is affected only if you have made drastic changes to Bugzilla (for example, if you have completely removed the "bug_status" column from the "bugs" table).
To apply the patch, follow these steps:
patch < p4dti-install-dir/bugzilla-2.10-patch
(where p4dti-install-dir is your P4DTI installation directory).You need to create a Bugzilla user for the replicator. The replicator uses e-mail addresses to work out which Perforce user corresponds to which Bugzilla user. A Perforce user that does not correspond to a Bugzilla user is translated to the replicator's Bugzilla user, except for user fields (for example, "AssignedTo") in jobs. The replicator rejects a change when there is no Bugzilla user corresponding to a changed user field.
To create a Bugzilla user for the replicator, follow these steps:
replicator_address). After patching the Bugzilla code, you need to enable the P4DTI extensions. To enable the extensions, follow these steps:
To disable use of the P4DTI from the Bugzilla user interface, switch the extensions off.
To start the replicator, follow these steps from the operating system command line:
python run.py. Alternatively, on Linux you may wish to start the script using the automatic startup script:
/etc/rc.d/init.d/p4dti startThe first time you start the replicator, it displays log output explaining how the replicator is setting up the defect tracker schema extensions, as shown in the following figure:
Figure 5. Example replicator log output on startup (TeamTrack integration)
2000-12-08 15:27:25 UTC P4DTI-00 replicator0 Reading SELECTIONS table to find type prefixes.
2000-12-08 15:27:25 UTC P4DTI-00 replicator0 Reading USERS table.
2000-12-08 15:27:27 UTC P4DTI-00 replicator0 Installing field 'P4DTI_RID' in the TS_CASES table.
2000-12-08 15:27:27 UTC P4DTI-00 replicator0 Installing field 'P4DTI_SID' in the TS_CASES table.
2000-12-08 15:27:27 UTC P4DTI-00 replicator0 Installing field 'P4DTI_JOBNAME' in the TS_CASES table.
2000-12-08 15:27:28 UTC P4DTI-00 replicator0 Installing field 'P4DTI_ACTION' in the TS_CASES table.
2000-12-08 15:27:30 UTC P4DTI-00 replicator0 Installed all new fields in the TS_CASES table.
2000-12-08 15:27:30 UTC P4DTI-00 replicator0 Put 'LAST_CHANGE' parameter in replicator configuration with value '23'
2000-12-08 15:27:30 UTC P4DTI-00 replicator0 Put 'SERVER' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': 'Perforce server 0'}"'
2000-12-08 15:27:31 UTC P4DTI-00 replicator0 Put 'STATUS_VALUES' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': '_new/assigned/closed/verified/deferred/cleared/clogged/pumped'}"'
2000-12-08 15:27:31 UTC P4DTI-00 replicator0 Put 'CHANGELIST_URL' parameter in replicator configuration with value '"{'sid': 'perforce0', 'description': None}"'
2000-12-08 15:27:45 UTC P4DTI-00 replicator0 Reading USERS table.
2000-12-08 15:27:45 UTC P4DTI-00 replicator0 Matched TeamTrack user 'pam' with Perforce user 'spam' by e-mail address 'gdr+pam@ravenbrook.com'.
2000-12-08 15:27:45 UTC P4DTI-00 replicator0 Matched TeamTrack user 'john' with Perforce user 'scone' by e-mail address 'gdr+john@ravenbrook.com'.
2000-12-08 15:27:45 UTC P4DTI-00 replicator0 Matched TeamTrack user 'fred' with Perforce user 'bedhead' by e-mail address 'gdr+fred@ravenbrook.com'. |
During its startup sequence, the replicator creates Perforce jobs corresponding to every defect tracker issue created or modified after the start date (start_date). It then polls for changes every poll_period seconds and replicates those changes. Figure 6 shows typical replicator log output when it is replicating a change.
Figure 6. Example replicator log output on replication (TeamTrack integration)
2000-12-08 15:59:12 UTC P4DTI-00 replicator0 1 issue has changed
2000-12-08 15:59:12 UTC P4DTI-00 replicator0 Set up issue '1' to replicate to job 'PRB00001'
2000-12-08 15:59:13 UTC P4DTI-00 replicator0 Replicating issue 'PRB00001' to job 'PRB00001'
2000-12-08 15:59:15 UTC P4DTI-00 replicator0 -- Changed fields: {'Engineer': 'fred', 'Resolution': '(None)', 'P4DTI-issue-id': '1', 'Owner': 'spam', 'Project': 'Pump', 'Description': 'I was just polishing the dome when my hand slipped.\012', 'Manager': 'spam', 'Priority': 'Urgent', 'How_Found': 'Normal_Use', 'Last_Modified_Date': '2000/12/08 15:59:05', 'P4DTI-rid': 'replicator0', 'Title': 'My oily rag fell into the chocolate pump', 'Workaround': '', 'Severity': 'High'} |
To stop the replicator, follow these steps:
If installed the P4DTI using the Linux RPM as described in section 4.3, "Linux installation", a startup script is automatically created in /etc/rc.d/init.d directory, so that the replicator starts when the machine is booted.
On Solaris or other Unixes, you might want to adapt the Linux startup script. It is in the file named startup-script in the installation directory.
On Windows, you must manually start the P4DTI whenever you reboot the machine. You might want to make this easier by creating a shortcut in the "Startup" folder of the "Start" menu of the Administrator.
Not all of the flexibility of the P4DTI is available using the configuration options described in this section. Advanced configuration of the P4DTI is possible, but beyond the scope of this manual. Here are some of the things that are possible with advanced configuration:
Contact Perforce technical support is you need any of these facilities.
You do not need to take any special action to migrate defect tracking data from your defect tracker to the integrated system. The replicator starts replicating defect tracker issues as soon as it starts up. Only issues that are created or modified after the start_date are replicated to Perforce.
Migrating your Perforce jobs to TeamTrack is not a straightforward operation. The strategy for migration is to convert Perforce jobs to TeamTrack, delete all the Perforce jobs, then replicate them back from TeamTrack. You'll need some TeamTrack expertise in order to set up TeamTrack to be ready for the migration. You'll need to work out how the fields in your Perforce jobs correspond to the fields in your TeamTrack cases. You'll need to do some Python programming to specify how the conversion should take place. If you have the necessary expertise, follow the steps below.
Warning. If you follow this procedure, you won't end up with identical jobs in Perforce to the ones you had to start with. The jobs will have different field names and different values in the fields. If you need to keep the field names and values the same, then this procedure won't work for you. What you want can be done but it requires you to extensively customize the P4DTI. See the P4DTI Integrator's Guide.
Take a copy of your Perforce jobs, just in case something goes wrong.
Set up TeamTrack projects and workflows so that you are ready to migrate. See chapters 5 and 6 of the tTrack 4.0 Administrator Manual. It is a good idea to work with TeamTrack for a while before migrating your data from Perforce, to make sure that your TeamTrack setup works well for you. If you have trouble, contact TeamShare support.
Configure the P4DTI as described in Section 5.1, but don't run it yet.
Edit the migrate_teamtrack.py
script. This script goes through all the jobs on your Perforce server,
converts them to TeamTrack cases, and submits them to TeamTrack. The
script won't work as delivered; you must edit it so that it does the
right thing. This is because only you know how the fields in your
Perforce jobs correspond to the fields in your TeamTrack cases so you
must specify how the conversion is to be done. Read the comments in the
code carefully. Test the script on a single job at first. If that job
is migrated wrongly, delete it in TeamTrack, fix the problem and try
again. If you have trouble, contact Perforce support.
When you're happy with your changes to the migrate_teamtrack.py script, run it on all your
jobs.
Check that your jobs have been migrated correctly.
If you're happy, delete all your jobs from Perforce, then run
the refresh.py script as described in Section 9.2, "Refreshing jobs in
Perforce".
Test the P4DTI configuration by creating a test issue and taking it through a complete life-cycle (that is, through the workflow) as described in the Perforce Defect Tracking Integration User's Guide. You might need to adapt the use cases described in the user's guide to your organization's workflow.
Test the P4DTI from both Perforce and the defect tracker. In Perforce, test the P4DTI using the interface that your developers are most likely to use. The main Perforce interfaces are:
To run the consistency checker and manage its output, follow these steps:
python check.py. You can also examine the database using a database application (for example, Microsoft Access or the mysql command) to ensure the Perforce data is in there.
You might want to provide training for Perforce and defect tracker users before they adopt the P4DTI for everyday use. If so, consider preparing training materials that walk them through the workflow for an issue, using the procedures that are documented in the Perforce Defect Tracking Integration User's Guide.
Even if you don't have a formal training session for your users, ensure that they:
You must stop and re-start the replicator as described in section 5.5, "Starting the replicator manually" after changing any of the configuration parameters described in section 5.1, "P4DTI configuration".
You must also stop and re-start the replicator after adding new users to your defect tracker or changing a user's userid or e-mail address. You might then need to edit jobs which mention that user in a field. The p4 jobs -e command can be used to search for text in the Perforce jobs.
You must also refresh Perforce jobs, as described in Section 9.2, "Refreshing jobs in Perforce", after changing either:
Perforce uses the field number in the jobspec to find data, not the field name (for more information, see the Perforce System Administrator's Guide). If you change the list of replicated fields, then the field numbers change, which means that the fields of existing jobs in Perforce will be mixed up. Refreshing the jobs re-creates them from the defect tracker with the correct fields.
Refreshing jobs deletes all the existing jobs in Perforce and replicates them from the defect tracker's database. This procedure is useful if:
To refresh the Perforce jobs, follow these steps from the operating system command line:
python refresh.py. python run.py. To uninstall the P4DTI, follow these steps:
/etc/rc.d, and so on. rpm -e p4dtiTo troubleshoot a problem with the P4DTI, follow these steps:
If you can't solve the problem, contact Perforce support (for details, see <http://www.perforce.com/perforce/support.html>). Provide the following information:
readme.txt that came with your P4DTI distribution to identify the release). p4 info" at the operating system command line. config.py file. The P4DTI is incompatible with the version of Bugzilla which you are running. You are running a very old version of Bugzilla, or have customized Bugzilla.
Some fields are always replicated. For details, see the replicated_fields parameter.
Remove the system fields from your list of replicated fields.
The replicated_fields parameter specifies a field which is not in Bugzilla.
Reduce the number of fields that you replicate by removing items from the replicated_fields parameter.
You are running a version of Bugzilla with different bug statuses from those in Bugzilla 2.10. The P4DTI has attempted to choose a sensible translation of these bug statuses to Perforce job states, but has failed. You may be able to fix this by changing the closed_state parameter. Otherwise you must modify your Bugzilla configuration.
The P4DTI chooses the names of states of Perforce jobs based on the status names in Bugzilla. It uses the following translation system:
closed_state is not None, the P4DTI translates this status to "closed" and translates "CLOSED" to "bugzilla_closed".For instance, if the parameter closed_state is 'RESOLVED', the P4DTI will use the following translation table for the default Bugzilla statuses:
| Bugzilla status | Perforce state |
|---|---|
| UNCONFIRMED | unconfirmed |
| NEW | bugzilla_new |
| ASSIGNED | assigned |
| RESOLVED | closed |
| VERIFIED | verified |
| CLOSED | bugzilla_closed |
| REOPENED | reopened |
Alternatively, if the parameter closed_state is 'CLOSED' or None, the P4DTI will use the following translation table for the default Bugzilla statuses:
| Bugzilla status | Perforce state |
|---|---|
| UNCONFIRMED | unconfirmed |
| NEW | bugzilla_new |
| ASSIGNED | assigned |
| RESOLVED | resolved |
| VERIFIED | verified |
| CLOSED | closed |
| REOPENED | reopened |
You are running a version of Bugzilla with different bug fields from those in Bugzilla 2.10, and are trying to replicate a field called "code". Perforce doesn't allow a job field called "code". You should remove the "code" field from the replicated_fields parameter or modify your Bugzilla configuration to rename the field.
Check the closed_state parameter. It must be a valid Bugzilla state.
An error message beginning "Bugzilla database error" indicates that the replicator has had an unexpected difficulty in accessing the Bugzilla database. Possibly there is a problem with MySQL or MySQLdb. Possibly you are running a version of Bugzilla which is incompatible with the P4DTI, or have customized Bugzilla in such a way that the P4DTI has become confused. Please contact Perforce support (see section 11.1).
The replicator Perforce user (p4_user parameter) has an e-mail address that does not match the replicator_address parameter. See section 5.2.1.
There is no Bugzilla user whose e-mail address matches the replicator_address parameter. See Section 5.4.2.
A Perforce user has changed the 'status' field of a bug in a way not permitted by Bugzilla. For instance, moving a bug directly from UNCONFIRMED to CLOSED. These transitions are not allowed in Bugzilla, and the replicator enforces that prohibition by rejecting such a change.
It is difficult but possible to cause this error by making more than one change to the status in rapid succession (between two consecutive replicator polls). The replicator can't tell if that has happened, so has to reject the change anyway.
A Perforce user has changed the long description text in some way other than appending to it. See Section 5.1.1.
A Perforce user has made a change to a field which the replicator treats as read-only. See Section 5.1.1.
A Perforce user has set a field, which corresponds to a numeric field in Bugzilla, to something which couldn't be converted to a number.
A Perforce user has changed a job's status to "duplicate".
When a bug is marked as a duplicate in Bugzilla, the number of the other bug is provided and a message identifying it is appended to the long description. The Perforce job interface provides no easy way of expressing this, so the replicator does not allow it.
You have changed a user field in a job to a Perforce user who does not have a Bugzilla user record. The replicator is unable to replicate that field back to Bugzilla.
A Perforce user has made a change which Bugzilla would not have permitted them to make.
Bugzilla has complex access controls which prohibit some users from making some changes to bugs. Perforce protections cannot express these controls so the replicator enforces these controls by rejecting changes to jobs which would not be permitted by Bugzilla.
A Perforce user has made a change to a bug which Bugzilla would not allow them to view or edit.
Bugzilla bugs can be grouped into "bug groups", which restrict the ability of users to view or edit them. Perforce protections cannot express these bug groups, so the replicator must enforce the Bugzilla restrictions by rejecting changes made by users outside the necessary bug group.
The dbms_user parameter is set incorrectly or the dbms_password parameter is set to None when a password is required.
The dbms_password parameter is set incorrectly.
The MySQL server on dbms_host doesn't serve a database whose name matches the dbms_database parameter.
A MySQL connection couldn't be established to the host given by the dbms_host parameter on either the port given by the dbms_port parameter or the default MySQL port 3306. Possible causes include:
The host given by the dbms_host parameter could not be located.
The connection to the MySQL server has been lost. The replicator will recover when the server connection is re-established.
(for example: "P4DTI configuration error: Configuration parameter '%s' must be a string").
Preliminary checking of the parameters set in config.py has found a problem. Correct the named parameter and restart the P4DTI.
You don't have a licence for the replicator. See Section 3.2.1, "Software prerequisites".
You will see this error if you changed the p4_user parameter
but didn't delete the old userid.
You haven't given the replicator permission to edit the Perforce jobspec. The replicator needs to have superuser privileges in Perforce. For instructions, see Section 5.2.1, "Creating a Perforce user for the replicator".
We've seen this error when our SMTP server was refusing connections.
Check your smtp_server parameter. Check that your SMTP
server is up and running.
This means that TeamTrack reported an error, but provided no information about the cause of the error. The Windows Application Log on the TeamTrack server machine often contains more information about why problems are occurring. Use the Event Viewer to example the Windows Application Log on that machine.
You haven't created a user in TeamTrack for the replicator (for instructions, see Section 5.3.2, "Creating a TeamTrack user for the replicator"), or else you've given the replicator incorrect values for the teamtrack_user and teamtrack_password parameters.
The replicator's TeamTrack user lacks the "Connect using the API" privilege. You need to use the TeamTrack Administrator to assign this privilege. See Section 5.3.2, "Creating a TeamTrack user for the replicator".
The P4DTI can't connect to the TeamTrack server. Check
that TeamTrack is up and running. Check your setting for the teamtrack_server parameter.
The P4DTI doesn't support all TeamTrack field types. One of the fields in your replicated_fields parameter has an unsupported type.
You can determine the list of supported types from the type_table in configure_teamtrack.py. The most notable unsupported type is "MULTIPLE_SELECTION", because Perforce does not provide any kind of multiple selection interface.
If you really need to have the field replicated, you have the following options:
replicator.translator to handle the field type (for the existing translators, see dt_teamtrack.py). For instructions on how to extend the P4DTI, and how to contribute your extensions back to the community, see the Perforce Defect Tracking Integration Integrator's Guide. See above for the corresponding Bugzilla configuration error.
You can specify a list of fields for the P4DTI to replicate into jobs; for details, see the replicated_fields parameter. This error means that the P4DTI couldn't find one of the fields in the list. This problem might happen if you change the set of fields in TeamTrack.
Double-check the field names you specified as the replicated_fields. If you're changing fields in TeamTrack, see Section 9, "Maintaining the P4DTI", for important information.
The P4DTI chooses the names of states of Perforce jobs based on the state names in TeamTrack. It uses the following mapping system:
Resolve this problem by making the state names distinct in TeamTrack. Do not use spaces at the beginning of state names.
Perforce uses the field "code" to pass internal status information to clients.
In TeamTrack, change the logical name of the field to something other than "code" by following these steps:
| 2000-08-10 | RB | Created placeholder. |
| 2000-09-11 | GDR | Added instructions for demonstrating the integration and notes on version 0.2. |
| 2000-09-20 | RB | Replaced demo instructions with full documentation outline from documentation plan. |
| 2000-10-15 | RB | Added installation and uninstallation sections, and other sections discussed in [RB 2000-10-07]. Removed parts specific to Ravenbrook Information System. |
| 2000-10-16 | RB | Merged with master sources and GDR's demonstration instructions for version 0.2. More edits required to make this consistent with the master sources. |
| 2000-10-19 | GDR | Updated to fix defects in release 0.3.1 [GDR 2000-10-17a] and release 0.3.2 [RB 2000-10-18b]. |
| 2000-11-25 | LMB | Removed "system" from title. Made lots of minor formatting and transition edits. Moved Glossary to end of document. Reorganized Section 4. |
| 2000-11-26 | RB | Improved prerequisites section. Added draft Bugzilla prerequisites. Formatted troubleshooting section. Updated version 0.3 references to version 0.4. |
| 2000-11-27 | RB | Added readership. Removed some false statements. |
| 2000-11-29 | GDR | Revised section 5 (configuration) to explain how to use the automatic configuration engine for TeamTrack. Moved material from sections 4 and 5 to make an appendix E for advanced configuration. Added section 4.6, a placeholder that will describe how to create a Perforce user for the replicator. The integration with TeamTrack now requires Python 2.0. |
| 2000-11-29 | RB | Corrected overview and improved replicator diagram. Changed prerequisites to point at Perforce 2000.2 beta release. Added proper text to Bugzilla prerequisites section. Cross-referenced to User's Guide. |
| 2000-11-29 | LMB | Changed "—" to "--" because the former doesn't display properly in Netscape. Made some minor edits in Sections 1-3. |
| 2000-11-30 | LMB | Corrected figure numbers in the text that were off by one. Finished editing the AG. Swapped round Sections D and E. Searched the doc for "dfn" tags and incorporated those terms into the glossary. Deleted the list in Section 4.1 and folded its single entry into the preceding sentence. Added a short note to Section 4.4 to the effect that if you're using IIS, you don't need to stop and restart the TeamTrack server. Added a note to Section 4.6 that we need to tell admins to make the P4DTI user a Perforce super user and add it to the "p4 protect" table if they're using it. |
| 2000-11-30 | RB | Added instructions to upgrade users Perforce clients and to stop using TeamShare SourceBridge. Told the administrator to check the Windows event log when things go wrong, because the TeamShare API doesn't tell the replicator about errors. |
| 2000-11-30 | GDR | Added comments to the example jobspec in section D.2, and fixed
the formatting. Added note saying that you may not have a field
called "code". Listed the TeamTrack workflows that won't work well.
Wrote advice on how to configure the integration. Added the changelist_url configuration parameter. |
| 2000-12-01 | RB | Moved TeamTrack and Bugzilla configuration sections into the Configuration chapter, after the P4DTI configuration instructions. Added basic Linux installation instructions. Rewrote sections of the configuration instructions to go with the new flow. Deleted section on switching TeamTrack databases. Updated registry editing and Team Track privilege instructions. Added instructions for creating a Perforce user for the replicator. Explained multiple transition limitation. Updated screenshots. |
| 2000-12-04 | GDR | Made list of configuration parameters in Section 5.2 consistent with the configuration file (by alphabetizing both lists). Added missing configuration parameter closed_state. Added note in section 5.2 about checking the configuration. |
| 2000-12-05 | RB | Changed figures to use "div" tags in line with the user manual and to allow more flexible use of material in figures. Added basic notes on Bugzilla configuration (more to come). |
| 2000-12-06 | RB | Added section 2.3 about supported platform configurations. |
| 2000-12-07 | GDR | Advised admin to make e-mail addresses or userids the same in TeamTrack and Perforce. Advised admin to restart when users are added or changed. |
| 2000-12-07 | RB | Removed "TeamTrack only" notice from "replicated_fields" configuration parameter heading. |
| 2000-12-08 | GDR | Documented translation of "ignore" state. Improved advice about what to do with a field called "code". Fixed table of contents. |
| 2000-12-08 | GDR | Documented refresh_perforce.py. |
| 2000-12-08 | GDR | Documented translation of "ignore" state. Improved advice about what to do with a field called "code". Fixed table of contents. |
| 2000-12-08 | GDR | Documented refresh_perforce.py. |
| 2000-12-08 | RB | Brought configuration section 5.2 up to date with Bugzilla configurator (see job000115). |
| 2000-12-08 | RB | Updated to match unified configuration file and related re-organization of sources. |
| 2000-12-11 | GDR | Noted that the changelist_url configuration parameter must be suitable for passing to sprintf(). |
| 2000-12-13 | RB | Updated for version 0.5 to cover Bugzilla supported platforms, configurations, and known issues. Added changelist_url for use with P4Web. |
| 2000-12-13 | RB | Moving the Python 1.5.2 sources and Linux RPM to the project imports. |
| 2000-12-15 | NB | Added verbose configuration item. |
| 2000-12-15 | NB | Added more chat about bugzilla_user. |
| 2000-12-15 | NB | More about e-mail addresses, python script names, and deleting jobs. |
| 2000-12-22 | LMB | Started improving the manual as agreed in e-mail from RB (Documentation preparation meeting with LMB, 2000-12-20). |
| 2001-01-01 | LMB | Continued improving the manual as agreed in e-mail from RB, modulo deleting things (Documentation preparation meeting with LMB, 2000-12-20). Removed spaces around em-dashes as per "Read Me First!" |
| 2001-01-02 | GDR | The recommendation for administrator experience (section 3) is more realistic. Moved text from Appendix D to the Integrator's Guide, and replaced with a reference. Replaced "company.com" with "company.domain" since the former exists. Fixed typos and improved wording to fix defects recorded in [GDR 2000-12-08b], [GDR 2000-12-08c], and [GDR 2000-12-09]. Made cross-references consistent. Added two error messages to section 13.2. Added figures 5 and 6 showing example log output. Added section 5.4.3 about providing field descriptions. |
| 2001-01-02 | LMB | Merged GDR's changes to master manuals with this branch. Got about two-thirds of the way through a heavy copyedit. |
| 2001-01-03 | LMB | Made some changes suggested by GDR. Finished copyedit; noticed a number of things I didn't catch in the copyedit, so I'm sure there are other problems. Verified links. Indicated which comments I thought had been dealt with, in case GDR has time to work on this document tomorrow. Did a once-over in IE5 to make sure I didn't break anything too badly in the last two days. |
| 2001-01-05 | LMB | Dealt with about half of the changes that TC@perforce suggested. |
| 2001-01-06 | LMB | Procedurized and listed everything, as per TC@perforce's suggestions. |
| 2001-01-07 | LMB | Edited references. Replaced Section 5.3.3. |
| 2001-01-07 | LMB | Removed extraneous title material and Sections A and B. Removed comments and sent them in an e-mail to p4dti-staff. Did final copyedit and finished prepping manual to send to TC@perforce. |
| 2001-01-20 | LMB | Incorporated TC's handwritten edits. Cut lots of stuff out of the glossary. |
| 2001-01-21 | LMB | Re-added author, date, and other title material, and Sections A and B. Added comments on what I'd been doing to the manual in the interim (not much). Edited some new material from GDR. As per GDR's e-mail of 2001-01-18, point 5, deleted Section C and moved the material in Section D to Section 5.3.3. Fixed XREFS to Section 5; checked XREFS to Sections 12, 13, C, and D. Added material on when you might want to refresh jobs. Edited Section 10 slightly. Checked that procedure lead-ins and lists meet TC's specifications. Copied in Nick's changes to the info on patching Bugzilla and to the closed_state and replicated_fields parameters. Deleted mentions of bugzilla_user. Corrected internal XREFS. Copied over information on log files from master sources. Ran the spelling checker. |
| 2001-01-22 | LMB | As per TC@perforce's request and GDR's instructions, added menu command for how to start the TeamTrack Administrator from Windows. Added upgrade instructions to Section 4. |
| 2001-01-29 | NB | Added bugzilla_directory configuration parameter. |
| 2001-02-01 | RB | Updated references to Perforce manuals to version 2000.2. Added steps to section 10, "Uninstalling the P4DTI", to undo all installation steps. Added missing information to section 9, "Maintaining the P4DTI", and clarified some of the existing instructions. |
| 2001-02-02 | RB | Implemented paper review comments of 2001-01-25 by TC@perforce. |
| 2001-02-12 | GDR | Added start_date configuration parameter. Renamed refresh_perforce.py to refresh.py. |
| 2001-02-13 | GDR | Added note about security of jobs. Added socket.error error message to section 11.2. Alphabetized messages in section 11.2. Added note about "localhost" not working as a value for the teamtrack_server configuration parameter. Indicated that administrator_address and smtp_server can be None. |
| 2001-02-14 | RB | Added Linux RPM instructions. |
| 2001-02-15 | RB | Fixed "zcat" to "gunzip -c" to fool-proof unpacking. |
| 2001-02-16 | RB | Added brief instructions for starting the P4DTI automatically. |
| 2001-02-16 | NB | Added section 6.1, on migrating sets of issues. Also add some poll_period and start_date references. |
| 2001-02-21 | GDR | Documented p4dti.reg in section 5.3.1. |
| 2001-02-22 | NB | Documented MySQLdb and MySQL versions in section 3.4.1. |
| 2001-02-22 | GDR | Gave instructions in section 11.1 on identifying the release that you're using. Added "Can't create a new user - over licence quota" to error messages in section 11.2. Wrote section 6.2 on migrating from Perforce jobs. |
| 2001-02-26 | NB | Added Bugzilla-related errors to section 11.2. |
| 2001-02-27 | GDR | Alphabetized the list of error messages in section 11.2. Gave full error messages, including prefixes, for each error. Improved error descriptions. |
| 2001-03-02 | RB | Added missing contents entries. Fixed some HTML errors and tidied up some other HTML. Transferred copyright to Perforce under their license. Added section on advanced configuration. |
| 2001-03-02 | GDR | Removed section 9.3 on tracking down TeamTrack errors section, since it's redundant with section 11.2. |
| 2001-03-05 | RB | Added e-mail messages to the list of things we'd like in bug reports. Added some missing "abbr" tags. |
| 2001-03-16 | GDR | Changed "daemon licence" to "background user licence" for consistency with Perforce's own terminology. Background user licences are available from Perforce Customer Service, not Technical Support. |
| changelist | An atomic change transaction in Perforce. | |
| background user license | A license for a process rather than a person. | |
| fix | A job that has been linked to a changelist. | |
| issue | A unit of work tracked by the defect tracker, for example, a bug, a change request, or an enhancement request. | |
| job | A user-defined unit of work tracked by Perforce. | |
| replicator | A process that copies (replicates) data between a defect tracker and a Perforce server in order to keep each one up to date with changes made in the other. Replication allows developers to do their routine defect resolution work entirely from their Perforce client, without needing to use the defect tracker's interface. It also allows developers to relate their changes to defect tracking issues. | |
| workflow | A set of rules that control who can do what to which issues. |
This document is copyright © 2001 Perforce Software, Inc. All rights reserved.
Redistribution and use of this document in any form, with or without modification, is permitted provided that redistributions of this document retain the above copyright notice, this condition and the following disclaimer.
This document is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the copyright holders and contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this document, even if advised of the possibility of such damage.