- <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
- <html xmlns="http://www.w3.org/1999/xhtml">
- <head>
- <meta http-equiv="content-type" content="application/xhtml+xml; charset=ISO-8859-1" />
- <meta name="GENERATOR" content="Quanta Plus" />
- <title>How to use p4convert-svn</title>
- <style type="text/css" media="screen">
- /* Space out the list entries a little for clarity */
- li {
- padding-bottom: 10pt;
- }
- h1,h2,h3,h4,h5,h6 {
- font-family: Arial, Helvetica, sans-serif;
- font-weight: bold;
- color: #333377;
- }
- h1 {
- font-size: 150%;
- text-decoration: underline;
- text-align: center;
- }
- h2 {
- font-size: 130%;
- border-top: 1px solid;
- padding-top: 1em;
- }
- h3 {
- font-size; 115%;
- font-style: italic;
- }
- pre {
- font-size: 90%;
- border: 1px solid;
- background-color: #EEEEEE;
- padding-top: 10px;
- }
- #vital {
- color: red;
- }
- table {
- border: 1px solid;
- background-color: #EEEEEE;
- }
- div.copyright {
- border-top: 1px solid;
- font-size: small;
- }
- div.ident {
- font-size: small;
- }
- div.section {
- width: 1000px;
- }
- ul.biglist {
- list-style-type: none;
- }
- pre {
- margin: 3;
- }
- </style>
- </head>
- <body>
- <h1>Users Guide to p4convert-svn</h1>
- <h2>Contents</h2>
- <div class="contents">
- <ol>
- <li><a href="#section1">Version Information</a></li>
- <li><a href="#section2">What is p4convert-svn for ?</a></li>
- <li><a href="#section3">Terms and Conditions</a></li>
- <li><a href="#section4">System Requirements</a></li>
- <li><a href="#section5">Caveats</a></li>
- <li><a href="#section6">Setup and Usage</a></li>
- <li><a href="#section7">Generating a Subversion Dump file</a></li>
- <li><a href="#section8">General Configuration</a></li>
- <li><a href="#section9">Selective and Incremental Conversion</a></li>
- <li><a href="#section9.1">Filtering Subversion paths</a></li>
- <li><a href="#section10">Changelist offset options</a></li>
- <li><a href="#section11">Unicode Support</a></li>
- <li><a href="#section12">Advanced Configuration</a></li>
- <li><a href="#section13">Running 'Import Mode'</a></li>
- <li><a href="#section14">Running 'Conversion Mode'</a></li>
- <li><a href="#section15">Post Conversion</a></li>
- <li><a href="#section17">Notes</a></li>
- <li><a href="#section18">Verification</a></li>
- <li><a href="#section19">Output and Logs</a></li>
- <li><a href="#section20">Errors</a></li>
- </ol>
- </div>
- <div class="section"> <a name="section1"></a>
- <h2>Version Information</h2>
- <p> This documentation covers p4convert-svn version 2013.1. </p>
- </div>
- <div class="section"> <a name="section2"></a>
- <h2>What is p4convert-svn for?</h2>
- The p4convert-svn conversion tool imports data from Subversion and
- reconstructs the file revision history in Perforce. The Subversion data is
- read from a dump file and can be imported in one of two ways:
- <blockquote>
- <h4>Import Mode (front door)</h4>
- <p> Subversion revisions are imported into a running Perforce Server.
- The Perforce Server could be new or contain existing data, but it must
- be running. Subversion revisions are added sequentially; file content
- is 'imported' and metadata such as file attributes, descriptions,
- dates and usernames are added to the Perforce changelist.</p>
- <p> The recommended use of 'Import Mode' is to start a new Perforce
- server to import each Subversion revision into a Perforce changelist.
- Using this method will produce a one-to-one mapping of Subversion
- revision numbers to Perforce changelists. The new Perforce server can
- be used in isolation to confirm the success of the migration and then
- merged with an existing Perforce Server using the 'perfmerge++' tool.</p>
- <h4>Convert Mode (back door)</h4>
- <p> SVN revisions are converted in full, creating a Perforce journal and
- set of archive files. Once replayed the resulting conversion is just
- as if the changes had always been in Perforce. The results can be
- merged into an existing Perforce Server using 'perfmerge++'. The
- Import mode is considered to be the safest method as all files are
- imported through a Perforce Client. However, the Convert mode is
- significantly faster and requires less user interaction.</p>
- <p>Convert mode is an advanced feature and should only be attempted by a
- Perforce expert familiar with replaying and upgrading databases.
- Incremental conversions are not supported using Convert mode; however
- the slower Import mode can be used after the a conversion to update
- the migration. </p>
- <h4>Incremental Updates (front door)<br />
- </h4>
- <p> Incremental updates are possible only using the Import mode where
- new revision from Subversion are imported into Perforce. However, no
- changes should be made to the import area of the Perforce Server or
- conflicts may occur.<br />
- </p>
- </blockquote>
- </div>
- <div class="section"> <a name="section3"></a>
- <h2>Terms and Conditions</h2>
- <p> p4convert-svn is supplied by Perforce Software in the hope that it
- will be useful. It is a support utility, not a Perforce product. All use
- of this software is at the user's own risk and subject to the following
- terms and conditions. </p>
- <pre> THIS SOFTWARE IS PROVIDED BY PERFORCE SOFTWARE, INC. 'AS IS' AND ANY
- EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
- WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
- DISCLAIMED. IN NO EVENT SHALL PERFORCE SOFTWARE, INC. BE LIABLE FOR ANY
- DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
- (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
- LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
- ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
- THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
- </pre> </div>
- <div class="section"> <a name="section4"></a>
- <h2>System Requirements</h2>
- <ol>
- <li>An unzipped Subversion dump file of the repository to be converted
- (generated without the --delta flag).</li>
- <li>System must have <span style="font-weight: bold;">Java SE Runtime
- 1.7.</span><br />
- </li>
- <li>System must have the Perforce Server binaries 2010.2 or greater.<br />
- </li>
- <li> [Import mode] A running Perforce Server (typically empty) with no
- pending changes in the conversion area. <br />
- </li>
- <li>[Convert mode] An empty Perforce Root directory for the generated
- archive and journal files. </li>
- </ol>
- <blockquote>
- <h4>Resource Tips:</h4>
- </blockquote>
- <blockquote>Running some conversions, particularly in Conversion Mode can
- be very memory intensive, as we keep a mapping of revisions and
- integrations in memory. For large conversions or conversions with
- excessive branching, more memory will needed by the JVM.<br />
- <br />
- The <code>--info </code>option will scan the Subversion dump file and
- report usage data to help estimate the required memory for Convert
- Mode. For example:<br />
- <pre>java -jar p4convert-svn.jar --info <path_to_svn_dumpfile></pre>
- <br />
- Use the <code><i>-Xmx</i></code> flags to increase the memory allocated
- to the JVM. For example, to allocate 64GB (65536MB) use:
- <pre><code>java -Xmx65536M -jar p4convert-svn.jar myConfig.cfg<br /></code></pre>
- </blockquote>
- <ol>
- </ol>
- </div>
- <div class="section"> <a name="section5"></a>
- <h2>Caveats</h2>
- <ol>
- <li>The following table lists the combinations of tested operating
- systems and Perforce Server versions.</li>
- <ul>
- <li>
- <table width="430" height="130" border="1">
- <tbody>
- <tr>
- <td><b>Operating System</b><br />
- </td>
- <td align="center"><b>10.2<br />
- </b></td>
- <td align="center"><b>13.1<br />
- </b></td>
- </tr>
- <tr>
- <td>Ubuntu 10.04.4<br />
- </td>
- <td align="center">supported<br />
- </td>
- <td align="center">supported</td>
- </tr>
- <tr>
- <td>Darwin 11.4.0<br />
- </td>
- <td align="center">supported</td>
- <td align="center">supported</td>
- </tr>
- <tr>
- <td>Windows 7x64<br />
- </td>
- <td align="center">supported</td>
- <td align="center">supported</td>
- </tr>
- <tr>
- <td>SunOS 5.10<br />
- </td>
- <td align="center">supported</td>
- <td align="center">supported</td>
- </tr>
- </tbody>
- </table>
- </li>
- <li>It is likely that other combinations will work, however it is not
- possible to test all combinations. <br />
- </li>
- <li>Please contact Perforce if you have any OS or version queries. <br />
- </li>
- </ul>
- <li>[Import mode] Symbolic links are not supported in Import mode on
- Windows 7x64.</li>
- <li>Perforce keyword expansions are different from the Subversion
- standard and may yield different results when synced.</li>
- </ol>
- </div>
- <div class="section"> <a name="section6"></a>
- <h2>Setup and Usage</h2>
- <p> Download the self-contained JAR from Perforce and check that Java SE
- runtime is installed with version 1.7 or greater. </p>
- For usage details, execute the jar with no options (this will generate a
- default configuration file):<br />
- <pre>java -jar p4convert-svn.jar<br /><br />WARNING: No configuration file specified...
- Usage:
- java -jar p4convert-svn.jar <config file>
- java -jar p4convert-svn.jar --version
- java -jar p4convert-svn.jar --info <dump file>
- java -jar p4convert-svn.jar --users <dump file>
- java -jar p4convert-svn.jar --extract rev.node <dump file>
- Generating default config file 'default.cfg'...</pre>
- <br />
- Create a configuration file based on the generated default configuration,
- then start the conversion by executing the jar with the configuration
- file. For example:<br />
- <pre>java -jar p4convert-svn.jar config.txt</pre>
- </div>
- <div class="section"> <a name="section7"></a>
- <h2>Generating a Subversion Dump file</h2>
- <p> Both the Import and Convert modes require a Subversion dumpfile as the
- history data source. A dumpfile can be generated in several different
- ways; two of which are detailed below: </p>
- <dl>
- <dt>Using the svnadmin command:</dt>
- <dd>
- <pre><code>svnadmin dump <var>local_repo_path </var> > dumpfile.dmp</code></pre>
- </dd>
- <dd>
- <p> Replace <var>local_repo_path</var> with the path to the
- Subversion repo. Do not use the <code>'--delta'</code> flag option
- as the results cannot be parsed by the conversion tool.</p>
- </dd>
- <dt>Using the remote dump command:</dt>
- <dd>
- <pre><code>rsvndump url > dumpfile.dmp</code></pre>
- </dd>
- <dd>
- <p> Replacing <code>url</code> with the remote Subversion server. For
- example: <code>http://</code> or <code>file:///</code> or <code>svn://</code>.
- Note that for large servers generating the dump file locally is
- significantly faster, you might also consider creating a local copy
- of the subversion depot using <a href="http://svn.apache.org/repos/asf/subversion/trunk/notes/svnsync.txt">svnsync</a>,
- before generating the dump file.</p>
- </dd>
- </dl>
- </div>
- <div class="section"> <a name="section8"></a>
- <h2>General Configuration</h2>
- The java conversion requires a configuration file to setup the necessary
- options. A default configuration file "default.cfg" can be generated by
- running the converter without a parameter.
- <pre>java -jar p4convert-svn.jar<br /><br />WARNING: No configuration file specified...
- Usage:
- java -jar p4convert-svn.jar <config file>
- java -jar p4convert-svn.jar --version
- java -jar p4convert-svn.jar --info <dump file>
- java -jar p4convert-svn.jar --users <dump file>
- java -jar p4convert-svn.jar --extract rev.node <dump file>
- Generating default config file 'default.cfg'...
- </pre> Once you have "<code>default.cfg</code>" copy it to "<code>yourconfig.cfg</code>"
- and then customize it as needed for the conversion.
- <h3>General configuration options common to both conversion modes.</h3>
- <ol>
- <li>Local path to Subversion dump file.
- <pre><code>com.p4convert.svn.dumpFile=/Users/pallen/MyDumpFile.dmp</code></pre>
- </li>
- <li>Conversion mode '<code>true</code>' for Import mode and '<code>false</code>'
- for Conversion mode
- <pre><code>com.p4convert.svn.mode=true</code></pre>
- </li>
- <li>Import depot path and depot to be created, e.g. <code>//import/...
- </code>
- <pre><code>com.p4convert.core.depotPath=import</code></pre>
- </li>
- <li>An optional sub path can be specified below the depot, e.g. <code>//import/sub/...</code>
- (Note that the path must end with a '/' even if no path is used)
- <pre><code>com.p4convert.core.subPath=sub/
- (Default: com.p4convert.core.subPath=/)</code></pre>
- </li>
- </ol>
- </div>
- <div class="section"> <a name="section9"></a>
- <h2>Selective and Incremental Conversions</h2>
- <ol>
- <li> The subversion revision ranges can be limited using the
- configuration options '<code>startRevision</code>' and '<code>endRevision</code>'.
- The default is to start at revision 1 and import all revisions,
- defined by setting '<code>endRevision</code>' to 0.
- <pre><code> com.p4convert.svn.start=1
- com.p4convert.svn.end=0
- </code></pre>
- </li>
- <li> Incremental conversions are possible using the <b>'</b>Import
- Mode'. It is important to take note of the last imported Subversion
- revision and start from the next revision for the subsequent import.
- For example: to import the first 100 subversion revisions, set the
- config options to:
- <pre><code> com.p4convert.svn.start=1
- com.p4convert.svn.end=100
- </code></pre> </li>
- <li> At the start of the import the following summary is displayed: (In
- this example only the first 100 revisions are imported out of 23000.)
- <pre><code> Importing Subversion ranges:
- start: 1
- end: 100
- last: 23000
- </code></pre> </li>
- <li> For the next incremental import (say another 100 revisions) set the
- revision ranges to...
- <pre><code> com.p4convert.svn.start=101
- com.p4convert.svn.end=200
- </code></pre> </li>
- </ol>
- </div>
- <div class="section"> <a name="section9.1"></a>
- <h2>Filtering Subversion paths</h2>
- The default behavior is not to exclude any path in Subversion; however for
- large repositories with many 'tags/' folders or situations where only part
- of a Subversion repository is to be converted you may wish to exclude
- certain Subversion paths. Subversion path exclusion is possible using two
- map files 'exclude.map' and 'include.map'.<br />
- <br />
- The filtering is based on matching the Subversion path to a regular
- expression in the map files. The 'exclude.map' file is processed first
- and if the pattern matches part of the path then that path is skipped.
- The 'include.map' file can be used to overlay the 'exclude.map' file
- re-adding paths that were skipped. Before a conversion can start the
- filters must be verified against the Dump file (typically this is fairly
- quick and a displays a progress indicator). <br />
- <br />
- The verification step is to prevent the situation where an excluded path
- is relied on at a later point in the history for a branch, copy or merge
- action. If such a situation is found the paths are logged and the
- excluded 'source path' is added to the 'issue.map' file. To resolve the
- issue the exclusion should be removed from the 'exclude.map' file or a
- regular expression, based on the 'issue.map' file, added to the
- 'include.map' file.<br />
- <br />
- For example; to exclude all Subversion tags in the folder 'tags/', create
- an exclusion map file 'exclude.map':<br />
- <pre># exclude Subversion tags:<br />^tags/.*</pre>
- Then start the conversion to verify the map:<br />
- <pre>pallen-mac:main$ java -jar dist/p4convert-svn.jar Config/foo.cfg
- loading ChangeMap: changeMap.txt
- loading TypeMap: types.map
- importing revisions: 1 to 20635 out of 20635
- exclude.map: ^tags/.*
- Verifying exclusion map...
- issue: tags/rel-1.0.14/api
- issue: tags/rel-1.0.14/sys
- issue: tags/rel-2.0.3<br />Issues found, saving issue map...
- Caught EXIT shutting down ...</pre>
- Looking at the reported issues the tags 'rel-1.0.14' and 'rel-2.0.3' have
- some actions that conflict with our exclusion, to resolve this simply add
- the exclusions to the 'include.map' file:<br />
- <pre># issues reported for tags/ folder<br />^tags/rel-1.0.14/.*<br />^tags/rel-2.0.3/.*<br /></pre>
- <ol>
- </ol>
- </div>
- <div class="section"> <a name="section10"></a>
- <h2>Changelist Offset options</h2>
- <ol>
- <li> The default offset is 0 (i.e. not offsetting). Offset is currently
- available for 'Convert Mode' and allows subversion revisions to be
- converted into Perforce changelists by a fixed offset:
- <pre><code>com.p4convert.adv.offset=0</code></pre>
- </li>
- <li> After the conversion a 'changeMap' file is appended locally
- containing a Subversion revision number to Perforce change-list
- mapping. The file name is configured by the option:
- <pre><code>com.p4convert.log.changeMap=changeMap.txt</code></pre>
- A changemap looks like this:
- <pre><code>
- # <Change>, <SVN revision>
- 1, 1
- 2, 2
- 3, 3
- ...
- </code></pre> </li>
- </ol>
- </div>
- <div class="section"> <a name="section11"></a>
- <h2>Unicode Support</h2>
- <blockquote> The following Unicode enable options apply to Unicode support
- for 'Import Mode' and 'Convert Mode'. The Charset options are only
- applicable to 'Import Mode', when translating a file through the
- Perforce client. In 'Convert Mode' archive files are always written in
- UTF-8 for a Unicode enabled Perforce server. <br />
- <br />
- Defaults (for non-Unicode servers):
- <pre><code>com.p4convert.p4.unicode=false
- com.p4convert.p4.charset=<none>
- </code></pre> A UTF8 client example: (for a full list see 'p4 help charset')
- <pre><code>com.p4convert.p4.unicode=true
- com.p4convert.p4.charset=utf8
- </code></pre> For Unicode conversions set the JVM arg:
- <pre><code>-Dfile.encoding=UTF-8</code></pre>
- Some Solaris and Linux conversions may need the locale set:
- <pre><code>export LC_ALL=en_GB.UTF-8
- </code></pre> Once a Perforce server is switched to Unicode enabled mode (<code>-xi</code>),
- all client workspaces need to define a character set. (For details see:
- <a href="http://kb.perforce.com/article/524">http://kb.perforce.com/article/524</a>).
- Note: A non-Unicode enabled Perforce Server will accept UTF16 files. </blockquote>
- <blockquote>
- <h4>Normalisation</h4>
- </blockquote>
- <blockquote>Platform Unicode normalisation is detected when the
- configuration file is generated, however it can be changed by setting
- the following configuration option to '<code>NFC</code>' or '<code>NFD</code>':<br />
- <pre><code>com.p4convert.svn.normalisation=NFD</code></pre>
- The default detection is based on the following:<br />
- <table width="380" height="130" border="1">
- <tbody>
- <tr>
- <td><b>Platform<br />
- </b> </td>
- <td><b>Normalisation<br />
- </b> </td>
- </tr>
- <tr>
- <td>Windows<br />
- </td>
- <td>NFC<br />
- </td>
- </tr>
- <tr>
- <td>Mac<br />
- </td>
- <td>NFD<br />
- </td>
- </tr>
- <tr>
- <td>*nix / *nux<br />
- </td>
- <td>NFC<br />
- </td>
- </tr>
- <tr>
- <td>Sun<br />
- </td>
- <td>NFC<br />
- </td>
- </tr>
- </tbody>
- </table>
- <h4> Subversion Properties</h4>
- <p>By default the converter parses Subversion properties as UTF-8
- strings. The conversion uses properties such as <code>svn:log</code>,
- <code>svn:author</code> for attributes in Perforce and must decode the
- byte sequence to UTF-8. In some data sets Windows users may have
- added high ASCII characters in one or more code pages. This release
- now supports a configuration option:</p>
- <pre><code>com.p4convert.svn.propTextType=UNKNOWN</code></pre>
- <p>The following strings denote the supported char-sets:</p>
- <table width="100%" border="1">
- <tbody>
- <tr>
- <td>UNKNOWN</td>
- <td>Big5</td>
- <td>Shift_JIS</td>
- <td> windows-1251</td>
- </tr>
- <tr>
- <td>US-ASCII</td>
- <td>ISO-2022-JP</td>
- <td>ISO-8859-1</td>
- <td>windows-1254</td>
- </tr>
- <tr>
- <td>UTF-8</td>
- <td>ISO-2022-CN</td>
- <td>ISO-8859-2</td>
- <td>windows-1252</td>
- </tr>
- <tr>
- <td>UTF-16BE</td>
- <td>ISO-2022-KR</td>
- <td> ISO-8859-5 </td>
- <td>windows-1256 </td>
- </tr>
- <tr>
- <td>UTF-16LE</td>
- <td>GB18030</td>
- <td>ISO-8859-6</td>
- <td>IBM424_rtl</td>
- </tr>
- <tr>
- <td> UTF-32BE</td>
- <td>EUC-JP</td>
- <td> ISO-8859-7</td>
- <td> IBM424_ltr</td>
- </tr>
- <tr>
- <td>UTF-32LE</td>
- <td>EUC-KR</td>
- <td> ISO-8859-8</td>
- <td>IBM420_rtl </td>
- </tr>
- <tr>
- <td>BINARY</td>
- <td>KOI8-R</td>
- <td>ISO-8859-9</td>
- <td> IBM420_ltr</td>
- </tr>
- </tbody>
- </table>
- <p>The first scan is always 'UTF-8' followed by the configuration
- option. BINARY implies a skip and the string <code><binary
- property></code> is inserted.</p>
- <p><br />
- </p>
- </blockquote>
- </div>
- <div class="section">
- <h2>Advanced Configuration</h2>
- <blockquote>
- <h4> Directory Properties </h4>
- The following options allow Subversion Directory Properties to be stored
- as versioned files in Perforce: <br />
- To enable this mode set the following property to '<code>true</code>'
- <pre><code>com.p4convert.svn.propEnabled=true</code></pre>
- To select the property name and format (Note: only '<code>ini</code>'
- mode is supported)
- <pre><code>com.p4convert.svn.propEncoding=ini
- com.p4convert.svn.propName=.svn.properties</code></pre>
- <h4>Empty changelists</h4>
- The following property will attempt to skip empty changes (where the
- change contains no revisions). This is typically the default behavior of
- the client or 'Import Mode' so is only really used in 'Convert Mode'.
- <pre><code>com.p4convert.adv.skipEmpty=false</code></pre>
- <h4>Username translation</h4>
- <p>A username map file ('<code>users.map</code>') can be generated using
- the '<code>--users</code>' option and then the right-hand-side
- modified with the new user name. The rename will only occur if the
- conversion tool finds the <code>users.map</code> file in the current
- working directory.</p>
- <h4>Binary file detection</h4>
- <p> </p>
- <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Binary
- files can be identified by adding their extensions to the type map
- file 'types.map'. The format is based on Perforce typemap spec,
- however it is limited to paths of the form '//....ext' (where .ext is
- the binary extension). <br />
- </p>
- <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br />
- </p>
- <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Default
- Type map (types.map):</p>
- <pre style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">TypeMap:
- binary //....zip
- binary //....gif
- binary //....png
- binary //....jpg
- binary //....dll
- binary //....class
- binary //....jar
- binary //....ecsfr</pre>
- <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;"><br />
- </p>
- <p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px;">Modification
- bits (+mxwlk) are supported and can be added using the type mapping.
- Binary detection though the type map is recommended as conversion is
- much faster. Binary files not identified in the type map will be
- scanned by the ICU4J libraries and if no text/Unicode match is found
- they are assumed to be binary. Warning: ICU4J may incorrectly
- identify small binary files as text creating sync issues on Windows
- clients.<br />
- </p>
- <p> </p>
- <h4>Changelist Description Format</h4>
- <p>The <code>logRevID</code> option can be used to reformat the
- Subversion revision descriptions to include the revision ID using the
- template: <br />
- </p>
- <ul>
- <li><code><rev></code> substituted with the Subversion revision</li>
- <li><code><description></code> substituted with the Subversion
- log<br />
- </li>
- </ul>
- <p>Default (as-is): <code>com.p4convert.svn.logRevID=<description></code></p>
- <h4>Case Sensitivity</h4>
- <p>The platform case sensitivity is detected when generating the
- configuration file. There is normally no reason to change this
- behavior from the detected defaults. Conversions between different
- platforms should be avoided especially when converting from a case
- sensitive environment (such as Linux) to a case insensitive
- environment (such as Windows). The advanced case handling options
- supported are set using one of the following options:<br />
- </p>
- <pre><code>com.p4convert.adv.caseMode=FIRST</code></pre>
- <ul>
- <li><code>NONE</code> - treat all paths as case sensitive (Linux).</li>
- <li><code>LOWER</code> - convert all paths to lower case</li>
- <li><code>UPPER</code> - convert all paths to upper case</li>
- <li><code>FIRST</code> - use the first encountered case combination
- (Windows)</li>
- </ul>
- <p>When using the Convert mode the generated Perforce archive files are
- based on the platforms case sensitivity. However on Linux platforms it
- can be useful to store archives files as if on a case insensitive
- server (<code>p4d -C1</code>). This can be simulated by setting the
- following option to 'true': </p>
- <pre><code>com.p4convert.adv.lowerCase=true</code></pre>
- <b>Note</b>: that if this option is set the path to the Perforce root
- directory, defined by <font color="#cc0000"><code>com.p4convert.adv.p4root</code></font>,
- must be in <font color="#cc0000">lower case</font> and the case mode of
- <font color="#cc0000"><code>FIRST</code></font> must be used.
- <h4>RCS Keyword expansion (<code>svn:keywords</code>)</h4>
- <p>By default RCS keyword expansion attributes are imported; however
- setting the configuration option:</p>
- <pre> <code>com.p4convert.svn.keepKeyword=true</code><code></code> </pre>
- <p>Setting a value of <code>false</code> will ignore all previous
- keyword attributes and import the files as normal text. See <a href="#keyword_notes">keyword
- notes</a> for known issues.</p>
- <h4>Merge Information (<code>svn:mergeinfo</code>)</h4>
- <p>Supports SVN 1.5-1.7 merge information and calculates the
- corresponding Perforce integration credit for the various actions.
- The feature is not enabled by default and if required the following
- configuration option must be set to true:</p>
- <pre>com.p4convert.svn.mergeInfoEnabled=true</pre>
- </blockquote>
- </div>
- <div class="section"> <a name="section13"></a>
- <h2>Running 'Import Mode'</h2>
- <ul>
- <li>Import mode will pull in file revisions from a Subversion dump file
- adding them to the Perforce Server specified by the connection details
- in the configuration options. To use this mode set '<code>com.p4convert.svn.mode</code>'
- to <code>true</code>.</li>
- <li>Care should be taken when adding data to a pre-existing Perforce
- Server that the revision actions do not conflict with revisions
- already in the Perforce Server; typically, this can occur if the
- Server has been in-use since the previous migration.</li>
- <li>To avoid such scenarios the import should either be to a unique
- depot, to avoid conflict, or the Perforce Server should be Read Only
- during subsequent migrations. After a conversion is complete, it is
- possible to merge the new data with an existing depot using tools like
- PerfMerge++.</li>
- <li>The converter will check that your Perforce Server has no pending
- changes, and will abort a conversion if any are detected</li>
- </ul>
- <h3>Configuration options specific to 'Import Mode':</h3>
- <ol>
- <li>Perforce server address and port (escape the '<code>:</code>' with '<code>\:</code>')
- <pre><code>com.p4convert.p4.port=localhost\:4444</code></pre>
- </li>
- <li>Default user and client for server connection
- <pre><code>com.p4convert.p4.client=svn-client
- com.p4convert.p4.user=svn-user
- </code></pre> </li>
- <li>Client workspace root used to import files into Perforce
- <pre><code>com.p4convert.p4.clientRoot=/Users/pallen/ws/</code></pre>
- </li>
- <li>If Security is set to level 1 or greater then the 'svn-user' must
- have 'admin' permissions in the Protection table and the password
- supplied as a string (the default is no string after the '=' sign). <br />
- <pre>com.p4convert.p4.passwd=PaSSwoRd</pre>
- </li>
- <li>Alternatively if the user is already logged in and there is a valid
- ticket, then leave the password field unset and <strong>set</strong>
- your environment for P4TICKETS (don't relay on the P4TICKETS unset
- default as p4-java will not find your ticket file).</li>
- </ol>
- </div>
- <div class="section"> <a name="section14"></a>
- <h2>Running 'Convert Mode'</h2>
- <ul>
- <li>Convert mode is more advanced and requires knowledge of the Perforce
- Journal replay and Archive file store.</li>
- <li>This mode can only be used for single shot conversions and cannot be
- used incrementally.</li>
- <li>The performance of conversion mode is significantly better than
- Import mode (x100 sometimes!).</li>
- <li>After a using the 'Convert mode' the administrator will need to run
- several commands to rebuild the Perforce server and upgrade the
- metadata. Please refer to the <a href="#section15">'Post Conversion'</a>
- section for step-by-step instructions.</li>
- <li>To use this mode set <b>'</b><code>com.p4convert.svn.mode</code><b>'</b>
- to <code>false</code>.</li>
- </ul>
- <h3>Configuration options specific to 'Conversion Mode':</h3>
- <ol>
- <li>Perforce server root address (path should end with a '<code>/</code>')<br />
- <pre><code>com.p4convert.adv.p4root=/full/path/to/p4_root/</code></pre>
- </li>
- <li>Change list offset (handy for batched conversions)
- <pre><code>com.p4convert.adv.offset=0</code></pre>
- </li>
- <li>Generated journal names (useful to increment the prefix when running
- batched conversions)
- <pre><code>com.p4convert.adv.jnlIndex=0
- com.p4convert.adv.jnlPrefix=jnl.</code></pre>
- </li>
- <li>Mimic the 2011.1 or greater credit behavior on rollbacks /
- downgrades (to enable set value to '<code>true</code>')
- <pre><code>com.p4convert.adv.downgrade=false</code></pre>
- </li>
- <li>Perforce normalises line-endings when storing the file on the server
- and restores them based on the client workspace options and platform
- type. However, in special cases it can be useful to store line-ending
- in the server and use the 'share' option in the client. To disable
- normal line-ending support set the following option to '<code>false</code>':
- <pre><code>com.p4convert.adv.lineEnding=true</code></pre>
- </li>
- <li>For non Unicode servers or to simplify storage of hi-ASCII files,
- setting following option to '<code>false</code>' will store the file
- as '<code>binary</code>':
- <pre><code>com.p4convert.p4.unicode=false</code></pre>
- </li>
- </ol>
- <p></p>
- </div>
- <div class="section"> <a name="section15"></a>
- <h2>Post Conversion [Conversion Mode]</h2>
- To finish the conversion you will need to install p4 and p4d and run a few
- Perforce commands.
- <ol>
- <li> [Required] Change directory to P4ROOT, check there are no db.*
- files present and then replay the journal file(s):
- <pre><code>$ cd p4_root
- $ p4d -r . -jr jnl.0
- Perforce db files in '.' will be created if missing...
- Recovering from jnl.0...
- Perforce server info:
- Server version 33 is replaying a version 0 journal/checkpoint.</code></pre>
- Note: the Server version is set to 0 to remind the administrator that
- an upgrade is required, see step 3.<br />
- </li>
- <li>or, for multiple journal files:
- <pre><code>$ p4d -r . -jr jnl.0 jnl.1 ....</code></pre>
- and with nohup:
- <pre><code>nohup p4d -r . -jr jnl.0 jnl.1 .... &</code></pre>
- </li>
- <li> [Required] Upgrade the database from 2004.2 schema. For simplicity
- the conversion generates a database using an old schema, allowing you
- to upgrade to a Perforce Server version of your choice. From the
- P4ROOT directory run the upgrade command:
- <pre><code>$ p4d -r . -xu
- Perforce db files in '.' will be created if missing...
- 2001.1: splitting db.integ into db.integed/db.resolve.
- 2001.1: splitting db.have into db.have and db.label.
- 2002.1: splitting pending db.change into db.changex.
- 2002.2: upgrading tempobj filetype in db.rev.
- 2002.2: upgrading tempobj filetype in db.working.
- 2003.1: initialize default depot.
- 2003.2: upgrading db.user.
- 2005.1: building db.revhx (headrev) table.
- 2005.1: building db.locks from db.working.
- 2005.2: building db.revdx (delrev) table.
- 2005.2: moving spec depot entries into db.revsx.
- 2007.3: (re)building haveMap from db.have/db.working.
- 2007.3: (re)building db.archmap (lazy-copy map) table.
- 2007.3: removing old db.archive.
- 2008.1: upgrading db.change.
- 2009.2: moving db.boddate/db.ixdate into db.bodtext/db.ixtext.
- 2009.2: removing db.boddate/db.ixdate.
- 2010.2: adding db.config.
- 2011.1: upgrading tiny.db.
- ...upgrades done</code></pre>
- </li>
- <li>[Optional] If the conversion was run in Unicode mode (where <code>com.p4convert.p4.unicode=true</code>)
- and users are going to continue to add Unicode content then you may
- wish to set the server to Unicode mode. To enable Unicode run the
- following command from the P4ROOT directory:
- <pre><code>$ p4d -r . -xi</code></pre>
- </li>
- <li>[Alternative] If you are running your Perforce Server on Linux and
- your user base is predominantly Windows you may wish to force the
- server to run as case insensitive (only allowing one version of case
- for paths and files). The conversion option <code>com.p4convert.adv.lowerCase=true</code>
- will have been used with the case mode set to <code>com.p4convert.adv.caseMode=FIRST</code>
- . All p4d commands must be include the '<code>-C1</code>' flag, this
- includes the earlier step 1 and step 2.<br />
- <p></p>
- For example:
- <pre><code>$ cd p4_root
- $ p4d <font color="#cc0000">-C1</font> -r . -jr jnl.0
- $ p4d <font color="#cc0000">-C1</font> -r . -xu</code></pre>
- </li>
- <li>[Recommended] Some archive files may not have MD5 sum digests
- (typically where Subversion did not store the digest or the digest
- does not match due to the use of keyword expansion). To fill in the
- metadata for MD5 sum digests and archive file sizes use the verify
- command (this might be best split into depots and sub directories for
- large servers):
- <pre><code>$ p4 verify -u //...</code></pre>
- </li>
- </ol>
- </div>
- <div class="section"> <a name="section17"></a>
- <h2>NOTES: </h2>
- <h3><a name="keyword_notes"></a>Keyword Expansion Issues:</h3>
- The conversion process preserves keyword file types and sets these files
- in Perforce with the +k modifier. However keywords in Subversion and
- Perforce are expanded differently. For example: In Subversion the keyword
- <b>$Revision$ </b>expands to:
- <pre><code>$Revision: 25005 $</code> (a change number in Perforce)</pre>
- Another example is the keyword <b>$Date$</b>, which in Subversion gives
- you the time as well...
- <pre><code>$Date: 2006-02-17 12:09:10 +0000 (Fri, 17 Feb 2006) $</code></pre>
- and in Perforce...
- <pre><code>$Date: 2006/02/17 $</code></pre>
- Here is a list of alternatives, (based on the above example):
- <pre>$Revision$ ==> $Change$ gives $Change: 25005 $<br />$Date$ ==> $DateTime$ gives $DateTime: 2006/02/17 12:09:10 $
- </pre>
- Due to the differences in keyword expansion the MD5 sum in Subversion is
- not valid for use in Perforce. This is why any keyword expanded files has
- an empty MD5 sum and why the <code>p4 verify -u //...</code> command is
- recommended in the 'Optional Steps' section. </div>
- <div class="section"> <a name="section18"></a>
- <h2>Verification</h2>
- Verification can be performed using a running Subversion server and
- comparing the differences in files in Perforce using the audit log. To
- enable the audit log set the following option to 'true' and choose the
- file name.
- <pre><code>
- com.p4convert.log.audit.enabled=true
- com.p4convert.log.audit.filename=audit.log
- </code></pre> A sample line of the audit log:
- <pre># <SVN path>, <SVN revision>, <P4 change>, <MD5 sum>
- trunk/src/foo.c, 1, 1, 1234567890abcdef1234567890abcdef
- </pre> </div>
- <div class="section"> <a name="section19"></a>
- <h2>Output and Logs</h2>
- <h3>Console output and logging configuration options</h3>
- The default SLF4J logging options can be overwritten with another
- configuration file; specify the <tt>log4j.configuration</tt> option with
- a local file, using the syntax <tt>file:<i>your_local_file</i></tt>.<br />
- <pre><code>java -Dlog4j.configuration=file:log4j.xml -jar p4convert-svn.jar <br /></code><code>
- </code></pre>Please use or refer to the sample 'debug.log4j.xml' for logging
- options.<br />
- <h3>These options are reserved for testing or future enhancements:</h3>
- <pre><code>
- com.p4convert.core.test
- com.p4convert.core.version
- com.p4convert.svn.emptyDirEnabled
- com.p4convert.svn.emptyDirName
- com.p4convert.adv.mimeTypes=false
- </code></pre>
- <h3> Reading Console/Logging output<br />
- </h3>
- <pre>Importing Subversion ranges:
- start: 1
- end: 23091
- last: 23091
- 1.0 A:D - trunk
- 1.1 A:D - trunk/src
- 1.2 A:F - trunk/src/foo.c (UTF-8)
- mapping: r1 => @1
- 2.0 A:F - trunk/src/bar.png (BINARY)
- mapping: r2 => @2
- ...
- </pre>
- <p> Explanation of output lines, for example: <code>1.2 A:F -
- trunk/src/foo.c (UTF-8)</code> </p>
- <p> The numbering '1.2' refers to the current Subversion revision and the
- node action. ('1' Subversion revision 1 and '.2' = the third node action
- as '.0' would be the first index) </p>
- <p> The letters 'A:F' refers to the Subversion action and if it is a file
- or directory operation. </p>
- <ul>
- <li>A Add</li>
- <li>B Branch</li>
- <li>E Edit</li>
- <li>I Integrate (merge fromNode)</li>
- <li>M Merge (svn:mergeinfo)</li>
- <li>C Copy (svn replace action with fromNode)</li>
- <li>U Update (svn replace action)</li>
- <li>R Remove (delete)</li>
- <li>F File</li>
- <li>D Directory</li>
- </ul>
- <p></p>
- The subversion path <code>'trunk/src/foo.c'</code> is followed by the
- detected type. <br />
- <blockquote>
- <table width="510" height="790" border="1">
- <tbody>
- <tr>
- <td align="center"><strong>Detected type</strong><br />
- </td>
- <td align="center"><strong>Perforce base type</strong><br />
- </td>
- </tr>
- <tr>
- <td>UTF-8<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>UTF-16BE<br />
- </td>
- <td>utf16<br />
- </td>
- </tr>
- <tr>
- <td>UTF-16LE<br />
- </td>
- <td>utf16<br />
- </td>
- </tr>
- <tr>
- <td>UTF-32BE<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>UTF-32LE<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>Shift_JIS<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-2022-JP<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>ISO-2022-CN<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>ISO-2022-KR<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>GB18030<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>EUC-JP<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>EUC-KR<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>Big5<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-1<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-2<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-5<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-6<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-7<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-8<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>windows-1251<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>windows-1254<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>windows-1256<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>KOI8-R<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>ISO-8859-9<br />
- </td>
- <td>Unicode<br />
- </td>
- </tr>
- <tr>
- <td>IBM424_rtl<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>IBM424_ltr<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>IBM420_rtl<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>IBM420_ltr<br />
- </td>
- <td>downgraded to binary<br />
- </td>
- </tr>
- <tr>
- <td>BINARY<br />
- </td>
- <td>binary<br />
- </td>
- </tr>
- </tbody>
- </table>
- </blockquote>
- Note: The detected type is based on language detection using ICU4J and not
- Subversion MIME or Perforce detection.<br />
- <p> Finally '<code>mapping: r223 => @223</code>' refers to the
- Subversion revision number ('<code>r223</code>') to the Perforce
- changelist number ('<code>@223</code>'). Typically 1:1 unless offset or
- merged against an live or pre-existing Perforce Server. </p>
- </div>
- <div class="section"> <a name="section20"></a>
- <h2>Errors</h2>
- In 'Import Mode' Perforce related 'p4-java' messages are reported as
- warnings and should be verified, for example:
- <pre><code>
- 46.3 A:F - repo/trunk/my.file
- WARNING: p4java: //import/repo/trunk/my.file - file(s) up-to-date.
- </code></pre> In 'Conversion Mode' errors are reported as 'Panics' and an
- exception is thrown.<br />
- <br />
- Problematic Subversion dump records can be extracted and sent to Perforce
- when it is not possible to send in the whole dumpfile. The extracted
- dumpfile only contains meta-data and the file content is removed and
- replaced with a block count.<br />
- <br />
- During an exception note the Subversion revision and node ID (the example
- above has a revision number of 46 and a node number of 3).<br />
- Then run the following command to extract the problematic record, for
- example:<br />
- <pre><code>$ java -jar dist/p4convert-svn.jar --extract 46.3 mysvndump.dump
- <br />searching for node: 46.3...
- <font color="#009900">Node-path: repo/trunk/my.file
- Node-kind: file
- Node-action: add
- Prop-content-length: 10
- Text-content-length: 308
- Text-content-md5: 6a339b6ccf2af72d77169ef29b98eb0b
- Content-length: 318
- <br />PROPS-END</font></code></pre>
- A file is then generated called '<code>node.46.3.dump</code>' and will
- contain the Subversion record to be sent to Perforce. (The content
- highlighted in green is the Subversion meta-data, also contained in the
- file).<br />
- <br />
- </div>
- <p> </p>
- <div class="copyright"> Copyright 2013, Perforce Software Inc. All rights
- reserved. </div>
- <p></p>
- </body>
- </html>
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#1 | 10152 | alan_petersen | Populate //guest/alan_petersen/p4convert/... from //guest/perforce_software/p4convert/....... « |
11 years ago | |
//guest/perforce_software/p4convert/p4convert-svn.html | |||||
#1 | 9807 | Paul Allen | Initial import of p4-convert (from change 894340) | 11 years ago |