<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<title>Test design</title>
</head>
<body bgcolor="#FFFFFF" text="#000000" link="#000099" vlink="#660066" alink="#FF0000">
<div align="center">
<p>
<a href="http://www.ravenbrook.com/">Ravenbrook</a> /
<a href="http://www.ravenbrook.com/project/">Projects</a> /
<a href="http://www.ravenbrook.com/project/p4dti/">Perforce Defect Tracking Integration</a> /
<a href="../../index.html">Version 2.0 Product Sources</a> /
<a href="../index.html">Design</a>
</p>
<p><i><a href="http://www.ravenbrook.com/project/p4dti/">Perforce Defect Tracking Integration Project</a></i></p>
<hr />
<h1>Test design</h1>
<address>
<a href="mailto:gdr@ravenbrook.com">Gareth Rees</a>,
<a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>,
2001-07-02
</address>
</div>
<h2><a id="section-1" name="section-1">1. Introduction</a></h2>
<p> This document describes the automated test cases for the P4DTI, how
they are designed, and how to run them. The purpose of the document is
to allow P4DTI developers to collaborate on test case development and so
improve the quality of the P4DTI. </p>
<p> The intended readership is P4DTI developers and people developing
new integrations using the integration kit. </p>
<p> This document is not confidential. </p>
<h2><a id="section-2" name="section-2">2. Test framework</a></h2>
<p>The automated tests use the Python unit test framework [<a
href="http://pyunit.sourceforge.net/">PyUnit</a>]. To use the framework
with these tests, put the module <code>unittest.py</code> from the
PyUnit distribution on Python's library path. New unit test scripts
should be called <code>test_<i>module</i>.py</code>, where
<code><i>module</i>.py</code> is the module being tested. Each test
script should define a function called <code>tests</code>, which returns
an object belonging to the <code>unittest.TestSuite</code> class. The
<code>run_tests.py</code> script finds all tests in the current
directory and runs them.</p>
<p>To run the unit test for the <code>message</code> module, run the
command <code>python test_message.py</code>.</p>
<p>To run the "wrapping" test case (only) from the unit test for the
<code>message</code> module, run the command <code>python
test_message.py wrapping</code>.</p>
<p>To run all the tests in this directory, run the command <code>python
run_tests.py</code>.</p>
<h2><a id="section-3" name="section-3">3. Test database</a></h2>
<p> This section specifies test databases. The test cases rely on
specific features of the test databases (e.g., presence of certain
fields) in order to test features of the P4DTI. </p>
<h3><a id="section-3.1" name="section-3.1">3.1. Bugzilla test database</a></h3>
<p> The Bugzilla test databases are stored as
<code>bugzilla-<i>VERSION</i>-mysqldump</code>, where
<code><i>VERSION</i></code> is the Bugzilla version (2.12, 2.14, etc).
Each is produced by running Bugzilla, building up a bugs database, and
using <code>mysqldump</code> to generate the file. </p>
<p> See [<a title="making a better bugzilla test database"
href="http://info.ravenbrook.com/mail/2001/10/16/14-49-45/0.txt">NB
2001-10-17</a>] for the design, noting the following additions: </p>
<ol>
<li> A user with spaces in their login name, to whom at least one bug
is assigned (regression test for <a
href="http://www.ravenbrook.com/project/p4dti/issue/job000289/">job000289</a>). </li>
</ol>
<h2><a id="section-4" name="section-4">4. Testing the TeamTrack integration</a></h2>
<pre>
TeamTrack test procedure
Gareth Rees, Ravenbrook Limited, 2001-02-26
1. Introduction
This document explains how to carry out a test procedure for the
Perforce/TeamTrack integration on Windows NT or Windows 2000.
The intended audience is P4DTI project staff.
2. Setting up a testing session
Use this procedure to start a fresh testing session with a new TeamTrack
server, Perforce server and P4DTI (but don't run the P4DTI yet).
1. Stop any running Perforce servers and TeamTrack servers.
2. Make a new dated directory like d:\p4dti-test\YYYY-MM-DD\p4d.
3. Start a Perforce server in that directory (p4d -p 0.0.0.0:1666 -r
d:\p4dti-test\YYYY-MM-DD\p4d). Add a license file.
4. Start P4Web in browse mode (p4web -p 127.0.0.1:1666 -b).
5. Copy teamtrack-test.mdb into d:\p4dti-test\YYYY-MM-DD, using the
TeamTrack Administrator. Call the new ODBC database called
something like "P4DTI YYYY-MM-DD"; connect to that database; start
the TeamTrack web server.
6. Configure and run the perforce_sample_data.py script.
7. Unpack the P4DTI release into d:\p4dti-test\YYYY-MM-DD\p4dti.
8. Copy the appropriate configuration file from this directory to
d:\p4dti-test\YYYY-MM-DD\p4dti\config.py.
3. Initial installation tests
1. Normal installation. Follow the setup procedure; start the
replicator. It should replicate all issues successfully.
2. Recent start date. Follow the setup procedure; set the start_date
parameter to the current time; start the replicator. The
replicator should replicate no issues.
Set the start_date parameter to an old date; restart the
replicator. The replicator should replicate the old issues.
job000112. Can't easily replicate by project
Set replicate_p to "return self['PROJECTID'] in [3,4]". Check
that only the programming projects are replicated.
4. Use cases and regression tests (users)
1. Follow the setup procedure; start the replicator. Check the
initial e-mail message to the administrator.
2. Run perforce_sample_data.py to generate some activity in Perforce.
3. Follow use cases in the User Guide.
job000006. TeamShare API error reporting is inadequate
Make an edit you don't have permission to. Make a transition you
don't have permission to.
job000024. "\012" in fixes table in TeamTrack
job000053. Implicit changes made by the DT don't get replicated back
job000063. Dates aren't replicated with fixes
job000087. Replicator doesn't enforce licences in TeamTrack
Make a edit in Perforce as a user who has no corresponding user
in TeamTrack.
job000107. You can't replicate an issue's project
job000109. Can't get at Perforce information from TeamTrack
job000133. You can't close a job by fixing it
job000146. Empty date fields in TeamTrack give the replicator a headache
Add "ESTTIMETOFIX" and "ACTTIMETOFIX" to the replicated_fields.
job000156. Pending changelists aren't clearly indicated as such
job000169. Change numbers are links in TeamTrack even when no
changelist URL has been specified
job000182. Elapsed time fields aren't replicated properly
job000188. tTrack integration fails with non-ASCII characters
job000223. Quote in change comment terminates display in TeamTrack
job000226. Newlines don't show up in change descriptions in TeamTrack
job000233. When you submit a new issue to TeamTrack it overwrites the issue
5. Use cases and regression tests (administrator)
job000108. Userids can't differ between TeamTrack and Perforce
job000140. Logical field name "code" not allowed in TeamTrack
job000141. Can't add to replicated_fields list
</pre>
<h2><a id="section-A" name="section-A">A. References</a></h2>
<table>
<tr valign="top">
<td>[<a id="ref-NB-2001-10-17" name="ref-NB-2001-10-17" href="http://info.ravenbrook.com/mail/2001/10/16/14-49-45/0.txt">NB 2001-10-17</a>]</td>
<td>
"making a better bugzilla test database" (e-mail message);
<a href="mailto:nb@ravenbrook.com">Nick Barnes</a>;
<a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>;
2001-10-17.
</td>
</tr>
<tr valign="top">
<td>[<a id="ref-RB-2000-08-10a" name="ref-RB-2000-08-10a" href="../../manual/ag/index.html">RB 2000-08-10a</a>]</td>
<td>
"Perforce Defect Tracking Integration Administrator's Guide";
<a href="mailto:rb@ravenbrook.com">Richard Brooksby</a>;
<a href="http://www.ravenbrook.com/">Ravenbrook Limited</a>;
2000-08-10.
</td>
</tr>
<tr valign="top">
<td>[<a id="ref-Shaw-2001-06-27" name="ref-Shaw-2001-06-27" href="http://info.ravenbrook.com/mail/2001/06/27/21-57-46/0.txt">Shaw 2001-06-27</a>]</td>
<td>
"Re: TeamTrack licenses" (e-mail message);
<a href="mailto:kelly.shaw@teamshare.com">Kelly Shaw</a>;
<a href="http://www.teamshare.com/">TeamShare</a>;
2001-06-27.
</td>
</tr>
</table>
<h2><a id="section-B" name="section-B">B. Document History</a></h2>
<table>
<tr valign="top">
<td>2001-07-02</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Created.</td>
</tr>
<tr valign="top">
<td>2001-07-03</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Everyone has to have "Connect using the API" privilege in TeamTrack 4.5 in order for lifecycle testing to work. In TeamTrack 4.5 you need to specify an owner for each state.</td>
</tr>
<tr valign="top">
<td>2001-10-19</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Added new fields in TeamTrack, references to test cases, basic description of Bugzilla test database.</td>
</tr>
<tr valign="top">
<td>2002-01-10</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Added instructions to set up a regression test for <a href="http://www.ravenbrook.com/project/p4dti/issue/job000426/">job000426</a>.</td>
</tr>
<tr valign="top">
<td>2002-01-15</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Added fixed-length text field for regression test for <a href="http://www.ravenbrook.com/project/p4dti/issue/job000442/">job000442</a>.</td>
</tr>
<tr valign="top">
<td>2002-01-29</td>
<td><a href="mailto:gdr@ravenbrook.com">GDR</a></td>
<td>Create user with spaces.</td>
</tr>
<tr valign="top">
<td>2003-06-02</td>
<td><a href="mailto:nb@ravenbrook.com">NB</a></td>
<td>Removed Teamtrack section as obsolete.</td>
</tr>
</table>
<hr />
<p><small>Copyright © 2001 Ravenbrook Limited. This document is provided "as is", without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this document. You may make and distribute verbatim copies of this document provided that you do not charge a fee for this document or for its distribution.</small></p>
<div align="center">
<p><code>$Id: //info.ravenbrook.com/project/p4dti/version/2.0/design/test/index.html#3 $</code></p>
<p>
<a href="http://www.ravenbrook.com/">Ravenbrook</a> /
<a href="http://www.ravenbrook.com/project/">Projects</a> /
<a href="http://www.ravenbrook.com/project/p4dti/">Perforce Defect Tracking Integration</a> /
<a href="../../index.html">Version 2.0 Product Sources</a> /
<a href="../index.html">Design</a>
</p>
</div>
</body>
</html>