<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="content-type" content="application/xhtml+xml; charset=ISO-8859-1" />
<meta name="GENERATOR" content="Quanta Plus" />
<title>How to use p4convert</title>
<style type="text/css" media="screen">
/* Space out the list entries a little for clarity */
li {
padding-bottom: 10pt;
}
h1,h2,h3,h4,h5,h6 {
font-family: Arial, Helvetica, sans-serif;
font-weight: bold;
color: #333377;
}
h1 {
font-size: 150%;
text-decoration: underline;
text-align: center;
}
h2 {
font-size: 130%;
border-top: 1px solid;
padding-top: 1em;
}
h3 {
font-size; 115%;
font-style: italic;
}
pre {
font-size: 90%;
border: 1px solid;
background-color: #EEEEEE;
padding-top: 10px;
}
#vital {
color: red;
}
table {
border: 1px solid;
background-color: #EEEEEE;
}
div.copyright {
border-top: 1px solid;
font-size: small;
}
div.ident {
font-size: small;
}
div.section {
width: 1000px;
}
ul.biglist {
list-style-type: none;
}
pre {
margin: 3;
}
</style>
</head>
<body>
<h1>Users Guide to p4convert</h1>
<h2>Contents</h2>
<div class="contents">
<ol>
<li><a href="#section1">Version Information</a></li>
<li><a href="#section2">What is p4convert for ?</a></li>
<li><a href="#section3">Terms and Conditions</a></li>
<li><a href="#section4">System Requirements</a></li>
<li><a href="#section5">Caveats</a></li>
<li><a href="#section6">Setup and Usage</a></li>
<li><a href="#section7">Generating a Subversion Dump file</a></li>
<li><a href="#section8">General Configuration</a></li>
<li><a href="#section9">Selective and Incremental Conversion</a></li>
<li><a href="#section9.1">Filtering Subversion paths</a></li>
<li><a href="#section10">Changelist offset options</a></li>
<li><a href="#section11">Unicode Support</a></li>
<li><a href="#section12">Advanced Configuration</a></li>
<li><a href="#section13">Running 'Import Mode'</a></li>
<li><a href="#section14">Running 'Conversion Mode'</a></li>
<li><a href="#section15">Post Conversion</a></li>
<li><a href="#section17">Notes</a></li>
<li><a href="#section18">Verification</a></li>
<li><a href="#section19">Output and Logs</a></li>
<li><a href="#section20">Errors</a></li>
</ol>
</div>
<div class="section"> <a name="section1"></a>
<h2>Version Information</h2>
<p> This documentation covers p4convert version 2013.1. </p>
</div>
<div class="section"> <a name="section2"></a>
<h2>What is p4convert for?</h2>
The p4convert conversion tool imports data from Subversion and
reconstructs the file revision history in Perforce. The Subversion data is
read from a dump file and can be imported in one of two ways:
<blockquote>
<h4>Import Mode (front door)</h4>
<p> Subversion revisions are imported into a running Perforce Server.
The Perforce Server could be new or contain existing data, but it must
be running. Subversion revisions are added sequentially; file content
is 'imported' and metadata such as file attributes, descriptions,
dates and usernames are added to the Perforce changelist.</p>
<p> The recommended use of 'Import Mode' is to start a new Perforce
server to import each Subversion revision into a Perforce changelist.
Using this method will produce a one-to-one mapping of Subversion
revision numbers to Perforce changelists. The new Perforce server can
be used in isolation to confirm the success of the migration and then
merged with an existing Perforce Server using the 'perfmerge++' tool.</p>
<h4>Convert Mode (back door)</h4>
<p> SVN revisions are converted in full, creating a Perforce journal and
set of archive files. Once replayed the resulting conversion is just
as if the changes had always been in Perforce. The results can be
merged into an existing Perforce Server using 'perfmerge++'. The
Import mode is considered to be the safest method as all files are
imported through a Perforce Client. However, the Convert mode is
significantly faster and requires less user interaction.</p>
<p>Convert mode is an advanced feature and should only be attempted by a
Perforce expert familiar with replaying and upgrading databases.
Incremental conversions are not supported using Convert mode; however
the slower Import mode can be used after the a conversion to update
the migration. </p>
<h4>Incremental Updates (front door)<br />
</h4>
<p> Incremental updates are possible only using the Import mode where
new revision from Subversion are imported into Perforce. However, no
changes should be made to the import area of the Perforce Server or
conflicts may occur.<br />
</p>
</blockquote>
</div>
<div class="section"> <a name="section3"></a>
<h2>Terms and Conditions</h2>
<p> p4convert is supplied by Perforce Software in the hope that it
will be useful. It is a support utility, not a Perforce product. All use
of this software is at the user's own risk and subject to the following
terms and conditions. </p>
<pre> THIS SOFTWARE IS PROVIDED BY PERFORCE SOFTWARE, INC. 'AS IS' AND ANY
EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL PERFORCE SOFTWARE, INC. BE LIABLE FOR ANY
DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
</pre> </div>
<div class="section"> <a name="section4"></a>
<h2>System Requirements</h2>
<ol>
<li>An unzipped Subversion dump file of the repository to be converted
(generated without the --delta flag).</li>
<li>System must have <span style="font-weight: bold;">Java SE Runtime
1.7.</span><br />
</li>
<li>System must have the Perforce Server binaries 2010.2 or greater.<br />
</li>
<li> [Import mode] A running Perforce Server (typically empty) with no
pending changes in the conversion area. <br />
</li>
<li>[Convert mode] An empty Perforce Root directory for the generated
archive and journal files. </li>
</ol>
<blockquote>
<h4>Resource Tips:</h4>
</blockquote>
<blockquote>Running some conversions, particularly in Conversion Mode can
be very memory intensive, as we keep a mapping of revisions and
integrations in memory. For large conversions or conversions with
excessive branching, more memory will needed by the JVM.<br />
<br />
The <code>--info </code>option will scan the Subversion dump file and
report usage data to help estimate the required memory for Convert
Mode. For example:<br />
<pre>java -jar p4convert.jar --info <path_to_svn_dumpfile></pre>
<br />
Use the <code><i>-Xmx</i></code> flags to increase the memory allocated
to the JVM. For example, to allocate 64GB (65536MB) use:
<pre><code>java -Xmx65536M -jar p4convert.jar myConfig.cfg<br /></code></pre>
</blockquote>
<ol>
</ol>
</div>
<div class="section"> <a name="section5"></a>
<h2>Caveats</h2>
<ol>
<li>The following table lists the combinations of tested operating
systems and Perforce Server versions.</li>
<ul>
<li>
<table width="430" height="130" border="1">
<tbody>
<tr>
<td><b>Operating System</b><br />
</td>
<td align="center"><b>10.2<br />
</b></td>
<td align="center"><b>13.1<br />
</b></td>
</tr>
<tr>
<td>Ubuntu 10.04.4<br />
</td>
<td align="center">supported<br />
</td>
<td align="center">supported</td>
</tr>
<tr>
<td>Darwin 11.4.0<br />
</td>
<td align="center">supported</td>
<td align="center">supported</td>
</tr>
<tr>
<td>Windows 7x64<br />
</td>
<td align="center">supported</td>
<td align="center">supported</td>
</tr>
<tr>
<td>SunOS 5.10<br />
</td>
<td align="center">supported</td>
<td align="center">supported</td>
</tr>
</tbody>
</table>
</li>
<li>It is likely that other combinations will work, however it is not
possible to test all combinations. <br />
</li>
<li>Please contact Perforce if you have any OS or version queries. <br />
</li>
</ul>
<li>[Import mode] Symbolic links are not supported in Import mode on
Windows 7x64.</li>
<li>Perforce keyword expansions are different from the Subversion
standard and may yield different results when synced.</li>
</ol>
</div>
<div class="section"> <a name="section6"></a>
<h2>Setup and Usage</h2>
<p> Download the self-contained JAR from Perforce and check that Java SE
runtime is installed with version 1.7 or greater. </p>
For usage details, execute the jar with no options (this will generate a
default configuration file):<br />
<pre>java -jar p4convert.jar<br /><br />WARNING: No configuration file specified...
Usage:
java -jar p4convert.jar <config file>
java -jar p4convert.jar --version
java -jar p4convert.jar --info <dump file>
java -jar p4convert.jar --users <dump file>
java -jar p4convert.jar --extract rev.node <dump file>
Generating default config file 'default.cfg'...</pre>
<br />
Create a configuration file based on the generated default configuration,
then start the conversion by executing the jar with the configuration
file. For example:<br />
<pre>java -jar p4convert.jar config.txt</pre>
</div>
<div class="section"> <a name="section7"></a>
<h2>Generating a Subversion Dump file</h2>
<p> Both the Import and Convert modes require a Subversion dumpfile as the
history data source. A dumpfile can be generated in several different
ways; two of which are detailed below: </p>
<dl>
<dt>Using the svnadmin command:</dt>
<dd>
<pre><code>svnadmin dump <var>local_repo_path </var> > dumpfile.dmp</code></pre>
</dd>
<dd>
<p> Replace <var>local_repo_path</var> with the path to the
Subversion repo. Do not use the <code>'--delta'</code> flag option
as the results cannot be parsed by the conversion tool.</p>
</dd>
<dt>Using the remote dump command:</dt>
<dd>
<pre><code>rsvndump url > dumpfile.dmp</code></pre>
</dd>
<dd>
<p> Replacing <code>url</code> with the remote Subversion server. For
example: <code>http://</code> or <code>file:///</code> or <code>svn://</code>.
Note that for large servers generating the dump file locally is
significantly faster, you might also consider creating a local copy
of the subversion depot using <a href="http://svn.apache.org/repos/asf/subversion/trunk/notes/svnsync.txt">svnsync</a>,
before generating the dump file.</p>
</dd>
</dl>
</div>
<div class="section"> <a name="section8"></a>
<h2>General Configuration</h2>
The java conversion requires a configuration file to setup the necessary
options. A default configuration file "default.cfg" can be generated by
running the converter without a parameter.
<pre>java -jar p4convert.jar<br /><br />WARNING: No configuration file specified...
Usage:
java -jar p4convert.jar <config file>
java -jar p4convert.jar --version
java -jar p4convert.jar --info <dump file>
java -jar p4convert.jar --users <dump file>
java -jar p4convert.jar --extract rev.node <dump file>
Generating default config file 'default.cfg'...
</pre> Once you have "<code>default.cfg</code>" copy it to "<code>yourconfig.cfg</code>"
and then customize it as needed for the conversion.
<h3>General configuration options common to both conversion modes.</h3>
<ol>
<li>Local path to Subversion dump file.
<pre><code>com.p4convert.svn.dumpFile=/Users/pallen/MyDumpFile.dmp</code></pre>
</li>
<li>Conversion mode '<code>true</code>' for Import mode and '<code>false</code>'
for Conversion mode
<pre><code>com.p4convert.svn.mode=true</code></pre>
</li>
<li>Import depot path and depot to be created, e.g. <code>//import/...
</code>
<pre><code>com.p4convert.core.depotPath=import</code></pre>
</li>
<li>An optional sub path can be specified below the depot, e.g. <code>//import/sub/...</code>
(Note that the path must end with a '/' even if no path is used)
<pre><code>com.p4convert.core.subPath=sub/
(Default: com.p4convert.core.subPath=/)</code></pre>
</li>
</ol>
</div>
<div class="section"> <a name="section9"></a>
<h2>Selective and Incremental Conversions</h2>
<ol>
<li> The subversion revision ranges can be limited using the
configuration options '<code>startRevision</code>' and '<code>endRevision</code>'.
The default is to start at revision 1 and import all revisions,
defined by setting '<code>endRevision</code>' to 0.
<pre><code> com.p4convert.svn.start=1
com.p4convert.svn.end=0
</code></pre>
</li>
<li> Incremental conversions are possible using the <b>'</b>Import
Mode'. It is important to take note of the last imported Subversion
revision and start from the next revision for the subsequent import.
For example: to import the first 100 subversion revisions, set the
config options to:
<pre><code> com.p4convert.svn.start=1
com.p4convert.svn.end=100
</code></pre> </li>
<li> At the start of the import the following summary is displayed: (In
this example only the first 100 revisions are imported out of 23000.)
<pre><code> Importing Subversion ranges:
start: 1
end: 100
last: 23000
</code></pre> </li>
<li> For the next incremental import (say another 100 revisions) set the
revision ranges to...
<pre><code> com.p4convert.svn.start=101
com.p4convert.svn.end=200
</code></pre> </li>
</ol>
</div>
<div class="section"> <a name="section9.1"></a>
<h2>Filtering Subversion paths</h2>
The default behavior is not to exclude any path in Subversion; however for
large repositories with many 'tags/' folders or situations where only part
of a Subversion repository is to be converted you may wish to exclude
certain Subversion paths. Subversion path exclusion is possible using two
map files 'exclude.map' and 'include.map'.<br />
<br />
The filtering is based on matching the Subversion path to a regular
expression in the map files. The 'exclude.map' file is processed first
and if the pattern matches part of the path then that path is skipped.
The 'include.map' file can be used to overlay the 'exclude.map' file
re-adding paths that were skipped. Before a conversion can start the
filters must be verified against the Dump file (typically this is fairly
quick and a displays a progress indicator). <br />
<br />
The verification step is to prevent the situation where an excluded path
is relied on at a later point in the history for a branch, copy or merge
action. If such a situation is found the paths are logged and the
excluded 'source path' is added to the 'issue.map' file. To resolve the
issue the exclusion should be removed from the 'exclude.map' file or a
regular expression, based on the 'issue.map' file, added to the
'include.map' file.<br />
<br />
For example; to exclude all Subversion tags in the folder 'tags/', create
an exclusion map file 'exclude.map':<br />
<pre># exclude Subversion tags:<br />^tags/.*</pre>
Then start the conversion to verify the map:<br />
<pre>pallen-mac:main$ java -jar dist/p4convert.jar Config/foo.cfg
loading ChangeMap: changeMap.txt
loading TypeMap: types.map
importing revisions: 1 to 20635 out of 20635
exclude.map: ^tags/.*
Verifying exclusion map...
issue: tags/rel-1.0.14/api
issue: tags/rel-1.0.14/sys
issue: tags/rel-2.0.3<br />Issues found, saving issue map...
Caught EXIT shutting down ...</pre>
Looking at the reported issues the tags 'rel-1.0.14' and 'rel-2.0.3' have
some actions that conflict with our exclusion, to resolve this simply add
the exclusions to the 'include.map' file:<br />
<pre># issues reported for tags/ folder<br />^tags/rel-1.0.14/.*<br />^tags/rel-2.0.3/.*<br /></pre>
<ol>
</ol>
</div>
<div class="section"> <a name="section10"></a>
<h2>Changelist Offset options</h2>
<ol>
<li> The default offset is 0 (i.e. not offsetting). Offset is currently
available for 'Convert Mode' and allows subversion revisions to be
converted into Perforce changelists by a fixed offset:
<pre><code>com.p4convert.adv.offset=0</code></pre>
</li>
<li> After the conversion a 'changeMap' file is appended locally
containing a Subversion revision number to Perforce change-list
mapping. The file name is configured by the option:
<pre><code>com.p4convert.log.changeMap=changeMap.txt</code></pre>
A changemap looks like this:
<pre><code>
# <Change>, <SVN revision>
1, 1
2, 2
3, 3
...
</code></pre> </li>
</ol>
</div>
<div class="section"> <a name="section11"></a>
<h2>Unicode Support</h2>
<blockquote> The following Unicode enable options apply to Unicode support
for 'Import Mode' and 'Convert Mode'. The Charset options are only
applicable to 'Import Mode', when translating a file through the
Perforce client. In 'Convert Mode' archive files are always written in
UTF-8 for a Unicode enabled Perforce server. <br />
<br />
Defaults (for non-Unicode servers):
<pre><code>com.p4convert.p4.unicode=false
com.p4convert.p4.charset=<none>
</code></pre> A UTF8 client example: (for a full list see 'p4 help charset')
<pre><code>com.p4convert.p4.unicode=true
com.p4convert.p4.charset=utf8
</code></pre> For Unicode conversions set the JVM arg:
<pre><code>-Dfile.encoding=UTF-8</code></pre>
Some Solaris and Linux conversions may need the locale set:
<pre><code>export LC_ALL=en_GB.UTF-8
</code></pre> Once a Perforce server is switched to Unicode enabled mode (<code>-xi</code>),
all client workspaces need to define a character set. (For details see:
<a href="http://kb.perforce.com/article/524">http://kb.perforce.com/article/524</a>).
Note: A non-Unicode enabled Perforce Server will accept UTF16 files. </blockquote>
<blockquote>
<h4>Normalisation</h4>
</blockquote>
<blockquote>Platform Unicode normalisation is detected when the
configuration file is generated, however it can be changed by setting
the following configuration option to '<code>NFC</code>' or '<code>NFD</code>':<br />
<pre><code>com.p4convert.svn.normalisation=NFD</code></pre>
The default detection is based on the following:<br />
<table width="380" height="130" border="1">
<tbody>
<tr>
<td><b>Platform<br />
</b> </td>
<td><b>Normalisation<br />
</b> </td>
</tr>
<tr>
<td>Windows<br />
</td>
<td>NFC<br />
</td>
</tr>
<tr>
<td>Mac<br />
</td>
<td>NFD<br />
</td>
</tr>
<tr>
<td>*nix / *nux<br />
</td>
<td>NFC<br />
</td>
</tr>
<tr>
<td>Sun<br />
</td>
<td>NFC<br />
</td>
</tr>
</tbody>
</table>
<h4> Subversion Properties</h4>
<p>By default the converter parses Subversion properties as UTF-8
strings. The conversion uses properties such as <code>svn:log</code>,
<code>svn:author</code> for attributes in Perforce and must decode the
byte sequence to UTF-8. In some data sets Windows users may have
added high ASCII characters in one or more code pages. This release
now supports a configuration option:</p>
<pre><code>com.p4convert.svn.propTextType=UNKNOWN</code></pre>
<p>The following strings denote the supported char-sets:</p>
<table width="100%" border="1">
<tbody>
<tr>
<td>UNKNOWN</td>
<td>Big5</td>
<td>Shift_JIS</td>
<td> windows-1251</td>
</tr>
<tr>
<td>US-ASCII</td>
<td>ISO-2022-JP</td>
<td>ISO-8859-1</td>
<td>windows-1254</td>
</tr>
<tr>
<td>UTF-8</td>
<td>ISO-2022-CN</td>
<td>ISO-8859-2</td>
<td>windows-1252</td>
</tr>
<tr>
<td>UTF-16BE</td>
<td>ISO-2022-KR</td>
<td> ISO-8859-5 </td>
<td>windows-1256 </td>
</tr>
<tr>
<td>UTF-16LE</td>
<td>GB18030</td>
<td>ISO-8859-6</td>
<td>IBM424_rtl</td>
</tr>
<tr>
<td> UTF-32BE</td>
<td>EUC-JP</td>
<td> ISO-8859-7</td>
<td> IBM424_ltr</td>
</tr>
<tr>
<td>UTF-32LE</td>
<td>EUC-KR</td>
<td> ISO-8859-8</td>
<td>IBM420_rtl </td>
</tr>
<tr>
<td>BINARY</td>
<td>KOI8-R</td>
<td>ISO-8859-9</td>
<td> IBM420_ltr</td>
</tr>
</tbody>
</table>
<p>The first scan is always 'UTF-8' followed by the configuration
option. BINARY implies a skip and the string <code><binary
property></code> is inserted.</p>
<p><br />
</p>
</blockquote>
</div>
<div class="section">
<h2>Advanced Configuration</h2>
<blockquote>
<h4> Directory Properties </h4>
The following options allow Subversion Directory Properties to be stored
as versioned files in Perforce: <br />
To enable this mode set the following property to '<code>true</code>'
<pre><code>com.p4convert.svn.propEnabled=true</code></pre>
To select the property name and format (Note: only '<code>ini</code>'
mode is supported)
<pre><code>com.p4convert.svn.propEncoding=ini
com.p4convert.svn.propName=.svn.properties</code></pre>
<h4>Empty changelists</h4>
The following property will attempt to skip empty changes (where the
change contains no revisions). This is typically the default behavior of
the client or 'Import Mode' so is only really used in 'Convert Mode'.
<pre><code>com.p4convert.adv.skipEmpty=false</code></pre>
<h4>Username translation</h4>
<p>A username map file ('<code>users.map</code>') can be generated using
the '<code>--users</code>' option and then the right-hand-side
modified with the new user name. The rename will only occur if the
conversion tool finds the <code>users.map</code> file in the current
working directory.</p>
<h4>Binary file detection</h4>
<p> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Binary
files can be identified by adding their extensions to the type map
file 'types.map'. The format is based on Perforce typemap spec,
however it is limited to paths of the form '//....ext' (where .ext is
the binary extension). <br />
</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br />
</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Default
Type map (types.map):</p>
<pre style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">TypeMap:
binary //....zip
binary //....gif
binary //....png
binary //....jpg
binary //....dll
binary //....class
binary //....jar
binary //....ecsfr</pre>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br />
</p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Modification
bits (+mxwlk) are supported and can be added using the type mapping.
Binary detection though the type map is recommended as conversion is
much faster. Binary files not identified in the type map will be
scanned by the ICU4J libraries and if no text/Unicode match is found
they are assumed to be binary. Warning: ICU4J may incorrectly
identify small binary files as text creating sync issues on Windows
clients.<br />
</p>
<p> </p>
<h4>Changelist Description Format</h4>
<p>The <code>logRevID</code> option can be used to reformat the
Subversion revision descriptions to include the revision ID using the
template: <br />
</p>
<ul>
<li><code><rev></code> substituted with the Subversion revision</li>
<li><code><description></code> substituted with the Subversion
log<br />
</li>
</ul>
<p>Default (as-is): <code>com.p4convert.svn.logRevID=<description></code></p>
<h4>Case Sensitivity</h4>
<p>The platform case sensitivity is detected when generating the
configuration file. There is normally no reason to change this
behavior from the detected defaults. Conversions between different
platforms should be avoided especially when converting from a case
sensitive environment (such as Linux) to a case insensitive
environment (such as Windows). The advanced case handling options
supported are set using one of the following options:<br />
</p>
<pre><code>com.p4convert.adv.caseMode=FIRST</code></pre>
<ul>
<li><code>NONE</code> - treat all paths as case sensitive (Linux).</li>
<li><code>LOWER</code> - convert all paths to lower case</li>
<li><code>UPPER</code> - convert all paths to upper case</li>
<li><code>FIRST</code> - use the first encountered case combination
(Windows)</li>
</ul>
<p>When using the Convert mode the generated Perforce archive files are
based on the platforms case sensitivity. However on Linux platforms it
can be useful to store archives files as if on a case insensitive
server (<code>p4d -C1</code>). This can be simulated by setting the
following option to 'true': </p>
<pre><code>com.p4convert.adv.lowerCase=true</code></pre>
<b>Note</b>: that if this option is set the path to the Perforce root
directory, defined by <font color="#cc0000"><code>com.p4convert.adv.p4root</code></font>,
must be in <font color="#cc0000">lower case</font> and the case mode of
<font color="#cc0000"><code>FIRST</code></font> must be used.
<h4>RCS Keyword expansion (<code>svn:keywords</code>)</h4>
<p>By default RCS keyword expansion attributes are imported; however
setting the configuration option:</p>
<pre> <code>com.p4convert.svn.keepKeyword=true</code><code></code> </pre>
<p>Setting a value of <code>false</code> will ignore all previous
keyword attributes and import the files as normal text. See <a href="#keyword_notes">keyword
notes</a> for known issues.</p>
<h4>Merge Information (<code>svn:mergeinfo</code>)</h4>
<p>Supports SVN 1.5-1.7 merge information and calculates the
corresponding Perforce integration credit for the various actions.
The feature is not enabled by default and if required the following
configuration option must be set to true:</p>
<pre>com.p4convert.svn.mergeInfoEnabled=true</pre>
</blockquote>
</div>
<div class="section"> <a name="section13"></a>
<h2>Running 'Import Mode'</h2>
<ul>
<li>Import mode will pull in file revisions from a Subversion dump file
adding them to the Perforce Server specified by the connection details
in the configuration options. To use this mode set '<code>com.p4convert.svn.mode</code>'
to <code>true</code>.</li>
<li>Care should be taken when adding data to a pre-existing Perforce
Server that the revision actions do not conflict with revisions
already in the Perforce Server; typically, this can occur if the
Server has been in-use since the previous migration.</li>
<li>To avoid such scenarios the import should either be to a unique
depot, to avoid conflict, or the Perforce Server should be Read Only
during subsequent migrations. After a conversion is complete, it is
possible to merge the new data with an existing depot using tools like
PerfMerge++.</li>
<li>The converter will check that your Perforce Server has no pending
changes, and will abort a conversion if any are detected</li>
</ul>
<h3>Configuration options specific to 'Import Mode':</h3>
<ol>
<li>Perforce server address and port (escape the '<code>:</code>' with '<code>\:</code>')
<pre><code>com.p4convert.p4.port=localhost\:4444</code></pre>
</li>
<li>Default user and client for server connection
<pre><code>com.p4convert.p4.client=svn-client
com.p4convert.p4.user=svn-user
</code></pre> </li>
<li>Client workspace root used to import files into Perforce
<pre><code>com.p4convert.p4.clientRoot=/Users/pallen/ws/</code></pre>
</li>
<li>If Security is set to level 1 or greater then the 'svn-user' must
have 'admin' permissions in the Protection table and the password
supplied as a string (the default is no string after the '=' sign). <br />
<pre>com.p4convert.p4.passwd=PaSSwoRd</pre>
</li>
<li>Alternatively if the user is already logged in and there is a valid
ticket, then leave the password field unset and <strong>set</strong>
your environment for P4TICKETS (don't relay on the P4TICKETS unset
default as p4-java will not find your ticket file).</li>
</ol>
</div>
<div class="section"> <a name="section14"></a>
<h2>Running 'Convert Mode'</h2>
<ul>
<li>Convert mode is more advanced and requires knowledge of the Perforce
Journal replay and Archive file store.</li>
<li>This mode can only be used for single shot conversions and cannot be
used incrementally.</li>
<li>The performance of conversion mode is significantly better than
Import mode (x100 sometimes!).</li>
<li>After a using the 'Convert mode' the administrator will need to run
several commands to rebuild the Perforce server and upgrade the
metadata. Please refer to the <a href="#section15">'Post Conversion'</a>
section for step-by-step instructions.</li>
<li>To use this mode set <b>'</b><code>com.p4convert.svn.mode</code><b>'</b>
to <code>false</code>.</li>
</ul>
<h3>Configuration options specific to 'Conversion Mode':</h3>
<ol>
<li>Perforce server root address (path should end with a '<code>/</code>')<br />
<pre><code>com.p4convert.adv.p4root=/full/path/to/p4_root/</code></pre>
</li>
<li>Change list offset (handy for batched conversions)
<pre><code>com.p4convert.adv.offset=0</code></pre>
</li>
<li>Generated journal names (useful to increment the prefix when running
batched conversions)
<pre><code>com.p4convert.adv.jnlIndex=0
com.p4convert.adv.jnlPrefix=jnl.</code></pre>
</li>
<li>Mimic the 2011.1 or greater credit behavior on rollbacks /
downgrades (to enable set value to '<code>true</code>')
<pre><code>com.p4convert.adv.downgrade=false</code></pre>
</li>
<li>Perforce normalises line-endings when storing the file on the server
and restores them based on the client workspace options and platform
type. However, in special cases it can be useful to store line-ending
in the server and use the 'share' option in the client. To disable
normal line-ending support set the following option to '<code>false</code>':
<pre><code>com.p4convert.adv.lineEnding=true</code></pre>
</li>
<li>For non Unicode servers or to simplify storage of hi-ASCII files,
setting following option to '<code>false</code>' will store the file
as '<code>binary</code>':
<pre><code>com.p4convert.p4.unicode=false</code></pre>
</li>
</ol>
<p></p>
</div>
<div class="section"> <a name="section15"></a>
<h2>Post Conversion [Conversion Mode]</h2>
To finish the conversion you will need to install p4 and p4d and run a few
Perforce commands.
<ol>
<li> [Required] Change directory to P4ROOT, check there are no db.*
files present and then replay the journal file(s):
<pre><code>$ cd p4_root
$ p4d -r . -jr jnl.0
Perforce db files in '.' will be created if missing...
Recovering from jnl.0...
Perforce server info:
Server version 33 is replaying a version 0 journal/checkpoint.</code></pre>
Note: the Server version is set to 0 to remind the administrator that
an upgrade is required, see step 3.<br />
</li>
<li>or, for multiple journal files:
<pre><code>$ p4d -r . -jr jnl.0 jnl.1 ....</code></pre>
and with nohup:
<pre><code>nohup p4d -r . -jr jnl.0 jnl.1 .... &</code></pre>
</li>
<li> [Required] Upgrade the database from 2004.2 schema. For simplicity
the conversion generates a database using an old schema, allowing you
to upgrade to a Perforce Server version of your choice. From the
P4ROOT directory run the upgrade command:
<pre><code>$ p4d -r . -xu
Perforce db files in '.' will be created if missing...
2001.1: splitting db.integ into db.integed/db.resolve.
2001.1: splitting db.have into db.have and db.label.
2002.1: splitting pending db.change into db.changex.
2002.2: upgrading tempobj filetype in db.rev.
2002.2: upgrading tempobj filetype in db.working.
2003.1: initialize default depot.
2003.2: upgrading db.user.
2005.1: building db.revhx (headrev) table.
2005.1: building db.locks from db.working.
2005.2: building db.revdx (delrev) table.
2005.2: moving spec depot entries into db.revsx.
2007.3: (re)building haveMap from db.have/db.working.
2007.3: (re)building db.archmap (lazy-copy map) table.
2007.3: removing old db.archive.
2008.1: upgrading db.change.
2009.2: moving db.boddate/db.ixdate into db.bodtext/db.ixtext.
2009.2: removing db.boddate/db.ixdate.
2010.2: adding db.config.
2011.1: upgrading tiny.db.
...upgrades done</code></pre>
</li>
<li>[Optional] If the conversion was run in Unicode mode (where <code>com.p4convert.p4.unicode=true</code>)
and users are going to continue to add Unicode content then you may
wish to set the server to Unicode mode. To enable Unicode run the
following command from the P4ROOT directory:
<pre><code>$ p4d -r . -xi</code></pre>
</li>
<li>[Alternative] If you are running your Perforce Server on Linux and
your user base is predominantly Windows you may wish to force the
server to run as case insensitive (only allowing one version of case
for paths and files). The conversion option <code>com.p4convert.adv.lowerCase=true</code>
will have been used with the case mode set to <code>com.p4convert.adv.caseMode=FIRST</code>
. All p4d commands must be include the '<code>-C1</code>' flag, this
includes the earlier step 1 and step 2.<br />
<p></p>
For example:
<pre><code>$ cd p4_root
$ p4d <font color="#cc0000">-C1</font> -r . -jr jnl.0
$ p4d <font color="#cc0000">-C1</font> -r . -xu</code></pre>
</li>
<li>[Recommended] Some archive files may not have MD5 sum digests
(typically where Subversion did not store the digest or the digest
does not match due to the use of keyword expansion). To fill in the
metadata for MD5 sum digests and archive file sizes use the verify
command (this might be best split into depots and sub directories for
large servers):
<pre><code>$ p4 verify -u //...</code></pre>
</li>
</ol>
</div>
<div class="section"> <a name="section17"></a>
<h2>NOTES: </h2>
<h3><a name="keyword_notes"></a>Keyword Expansion Issues:</h3>
The conversion process preserves keyword file types and sets these files
in Perforce with the +k modifier. However keywords in Subversion and
Perforce are expanded differently. For example: In Subversion the keyword
<b>$Revision$ </b>expands to:
<pre><code>$Revision: 25005 $</code> (a change number in Perforce)</pre>
Another example is the keyword <b>$Date$</b>, which in Subversion gives
you the time as well...
<pre><code>$Date: 2006-02-17 12:09:10 +0000 (Fri, 17 Feb 2006) $</code></pre>
and in Perforce...
<pre><code>$Date: 2006/02/17 $</code></pre>
Here is a list of alternatives, (based on the above example):
<pre>$Revision$ ==> $Change$ gives $Change: 25005 $<br />$Date$ ==> $DateTime$ gives $DateTime: 2006/02/17 12:09:10 $
</pre>
Due to the differences in keyword expansion the MD5 sum in Subversion is
not valid for use in Perforce. This is why any keyword expanded files has
an empty MD5 sum and why the <code>p4 verify -u //...</code> command is
recommended in the 'Optional Steps' section. </div>
<div class="section"> <a name="section18"></a>
<h2>Verification</h2>
Verification can be performed using a running Subversion server and
comparing the differences in files in Perforce using the audit log. To
enable the audit log set the following option to 'true' and choose the
file name.
<pre><code>
com.p4convert.log.audit.enabled=true
com.p4convert.log.audit.filename=audit.log
</code></pre> A sample line of the audit log:
<pre># <SVN path>, <SVN revision>, <P4 change>, <MD5 sum>
trunk/src/foo.c, 1, 1, 1234567890abcdef1234567890abcdef
</pre> </div>
<div class="section"> <a name="section19"></a>
<h2>Output and Logs</h2>
<h3>Console output and logging configuration options</h3>
The default SLF4J logging options can be overwritten with another
configuration file; specify the <tt>log4j.configuration</tt> option with
a local file, using the syntax <tt>file:<i>your_local_file</i></tt>.<br />
<pre><code>java -Dlog4j.configuration=file:log4j.xml -jar p4convert.jar <br /></code><code>
</code></pre>Please use or refer to the sample 'debug.log4j.xml' for logging
options.<br />
<h3>These options are reserved for testing or future enhancements:</h3>
<pre><code>
com.p4convert.core.test
com.p4convert.core.version
com.p4convert.svn.emptyDirEnabled
com.p4convert.svn.emptyDirName
com.p4convert.adv.mimeTypes=false
</code></pre>
<h3> Reading Console/Logging output<br />
</h3>
<pre>Importing Subversion ranges:
start: 1
end: 23091
last: 23091
1.0 A:D - trunk
1.1 A:D - trunk/src
1.2 A:F - trunk/src/foo.c (UTF-8)
mapping: r1 => @1
2.0 A:F - trunk/src/bar.png (BINARY)
mapping: r2 => @2
...
</pre>
<p> Explanation of output lines, for example: <code>1.2 A:F -
trunk/src/foo.c (UTF-8)</code> </p>
<p> The numbering '1.2' refers to the current Subversion revision and the
node action. ('1' Subversion revision 1 and '.2' = the third node action
as '.0' would be the first index) </p>
<p> The letters 'A:F' refers to the Subversion action and if it is a file
or directory operation. </p>
<ul>
<li>A Add</li>
<li>B Branch</li>
<li>E Edit</li>
<li>I Integrate (merge fromNode)</li>
<li>M Merge (svn:mergeinfo)</li>
<li>C Copy (svn replace action with fromNode)</li>
<li>U Update (svn replace action)</li>
<li>R Remove (delete)</li>
<li>F File</li>
<li>D Directory</li>
</ul>
<p></p>
The subversion path <code>'trunk/src/foo.c'</code> is followed by the
detected type. <br />
<blockquote>
<table width="510" height="790" border="1">
<tbody>
<tr>
<td align="center"><strong>Detected type</strong><br />
</td>
<td align="center"><strong>Perforce base type</strong><br />
</td>
</tr>
<tr>
<td>UTF-8<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>UTF-16BE<br />
</td>
<td>utf16<br />
</td>
</tr>
<tr>
<td>UTF-16LE<br />
</td>
<td>utf16<br />
</td>
</tr>
<tr>
<td>UTF-32BE<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>UTF-32LE<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>Shift_JIS<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-2022-JP<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>ISO-2022-CN<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>ISO-2022-KR<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>GB18030<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>EUC-JP<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>EUC-KR<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>Big5<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-1<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-2<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-5<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-6<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>ISO-8859-7<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-8<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>windows-1251<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>windows-1254<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>windows-1256<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>KOI8-R<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>ISO-8859-9<br />
</td>
<td>Unicode<br />
</td>
</tr>
<tr>
<td>IBM424_rtl<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>IBM424_ltr<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>IBM420_rtl<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>IBM420_ltr<br />
</td>
<td>downgraded to binary<br />
</td>
</tr>
<tr>
<td>BINARY<br />
</td>
<td>binary<br />
</td>
</tr>
</tbody>
</table>
</blockquote>
Note: The detected type is based on language detection using ICU4J and not
Subversion MIME or Perforce detection.<br />
<p> Finally '<code>mapping: r223 => @223</code>' refers to the
Subversion revision number ('<code>r223</code>') to the Perforce
changelist number ('<code>@223</code>'). Typically 1:1 unless offset or
merged against an live or pre-existing Perforce Server. </p>
</div>
<div class="section"> <a name="section20"></a>
<h2>Errors</h2>
In 'Import Mode' Perforce related 'p4-java' messages are reported as
warnings and should be verified, for example:
<pre><code>
46.3 A:F - repo/trunk/my.file
WARNING: p4java: //import/repo/trunk/my.file - file(s) up-to-date.
</code></pre> In 'Conversion Mode' errors are reported as 'Panics' and an
exception is thrown.<br />
<br />
Problematic Subversion dump records can be extracted and sent to Perforce
when it is not possible to send in the whole dumpfile. The extracted
dumpfile only contains meta-data and the file content is removed and
replaced with a block count.<br />
<br />
During an exception note the Subversion revision and node ID (the example
above has a revision number of 46 and a node number of 3).<br />
Then run the following command to extract the problematic record, for
example:<br />
<pre><code>$ java -jar dist/p4convert.jar --extract 46.3 mysvndump.dump
<br />searching for node: 46.3...
<font color="#009900">Node-path: repo/trunk/my.file
Node-kind: file
Node-action: add
Prop-content-length: 10
Text-content-length: 308
Text-content-md5: 6a339b6ccf2af72d77169ef29b98eb0b
Content-length: 318
<br />PROPS-END</font></code></pre>
A file is then generated called '<code>node.46.3.dump</code>' and will
contain the Subversion record to be sent to Perforce. (The content
highlighted in green is the Subversion meta-data, also contained in the
file).<br />
<br />
</div>
<p> </p>
<div class="copyright"> Copyright 2013, Perforce Software Inc. All rights
reserved. </div>
<p></p>
</body>
</html>