Configuration
The java conversion requires a configuration file to setup the necessary
options. A default configuration file default.cfg
can
be generated by running the converter with the --default
parameter and specifying CVS
or SVN
.
For example: to generate a default configuration file for CVS, run...
java -jar p4convert.jar --type=CVS --default
Once you have default.cfg
, copy it to
yourconfig.cfg
and then customize it as needed for
the conversion.
Core converter settings.
The following settings appear at the top of the configuration file and are automatically defined at the time of generation. The Core converter settings should not be modified.
-
Configuration file schema version. Used to identify older configuration formats.
com.p4convert.core.schema=
5.50
-
SCM conversion type. Either set SVN or CVS configuration options.
com.p4convert.core.scmType=
SVN
-
Reserved for testing framework. Must be set to
false
for normal operation.com.p4convert.core.test=
false
-
Conversion tool's release version.
com.p4convert.core.version=
PUBLIC.10690
General Perforce converter settings.
-
Set the conversion mode to
IMPORT
for Import Mode and toCONVERT
for Conversion Mode.com.p4convert.p4.mode=
IMPORT
-
Import depot path and depot to be created, e.g.
//
import
/...com.p4convert.core.depotPath=
import
-
An optional sub path can be specified below the depot, e.g.
//import/
(Note that the path must end with a slash (sub
/.../
) even if no path is used). The default value is/
.com.p4convert.core.subPath=
sub
/
General Subversion converter settings.
If the configuration file has been correctly generated using the
--type=SVN --default
the Subversion configuration
options will appear as a block using the com.p4convert.svn
name space. These settings will only apply if the SCM
mode is set to SVN
:
com.p4convert.core.scmType=SVN
-
Local path to Subversion dump file.
com.p4convert.svn.dumpFile=
/Users/bruno/MyDumpFile.dmp
General CVS converter settings.
If the configuration file has been correctly generated using the
--type=CVS --default
the CVS configuration
options will appear as a block using the com.p4convert.cvs
name space. These settings will only apply if the SCM
mode is set to CVS
:
com.p4convert.core.scmType=CVS
-
Local path to CVSROOT files. (A deeper path is permitted to limit the scope of the import).
com.p4convert.cvs.cvsroot=
/Users/bruno/CVSROOT/
-
(Optionally) specify a CVS module to import from within the CVSROOT.
com.p4convert.cvs.cvsmodule=
projX
-
CVS time window. When no
commitid
is defined in the RCS file, the converter will calculate which revisions should be grouped into a changelist. The time window specifies the maximum range the converter should look ahead for matching revisions. (milliseconds)com.p4convert.cvs.timeWindow=
20000
-
CVS temporary directory. All the CVS assets are reformatted into full text files. The specified temporary directory is used to store the content during conversion.
com.p4convert.cvs.tmpDir=
tmp
Mapping CVS Paths
CVS paths are mapped into Perforce using the original RCS file structure, but prefixed with the branch name. The converter uses a branch name 'main' for the initial 1.1 branch and the symbol names for subsequent branches.
If requires an optional path map can be provided to remap the CVS path to a new structure within Perforce. When designing a path map, care must be taken to insure that all CVS paths have a destination mapping.
-
To enable CVS path mapping create a file called
path.map
with regex and group match. -
Only the first matching entry is used.
-
The regex and group match are seperated by ', ' (or in regex terms ',\s+').
-
Lines starting with '#' are ignored.
For example, 'trunk' is renamed to 'main', but other entries are left as-is.
# path.map
trunk/(.*), //import/main/{1}
(.*), //import/{1}
Note: if no file is found the default 'depot' and 'subPath' options are used to generate the map, preserving the original behaviour. CVS paths will always stat with the 'branch' name. 'main' for 1.1 and the symbol for other branches.
# path.map
main/projA/(.*), //import/projA/MAIN/{1}
release_(.*)/projA/(.*), //import/projA/REL{1}/{2}
(.*)/projA/(.*), //import/projA/TAG-{1}/{2}
(.*), //import/unexpected/{1}
Note
Adding a catch all is a good idea (as shown in the last line with the 'unexpected' directory).
Labeling CVS Paths
CVS will tag revisions with symbols, some of these symbols may refer to labels, whilst others are branch names. If labeling is disabled (default), then only symbols with history are converted to Perforce branches, the remaining symbols are ignored. However, if labels are enabled then the remaining symbols from the branch detection are converted to Perforce Labels.
-
To enable labeling of CVS symbols tags, set the
com.p4convert.cvs.labels
totrue
. -
In multiple CVS conversions there is the potential for namespace clashing if the label has been previously used. By default, the converter will use the CVS symbol as the name for the Perforce label. However, this can be formatted by setting the following option:
com.p4convert.cvs.labelFormat
. The parameter{symbol}
will be substituted by the CVS symbol name.For example:
com.p4convert.cvs.labels=true com.p4convert.cvs.labelFormat=import_{symbol}_label
Will import a CVS symbol called
REL1.0
as a Perforce labelimport_REL1.0_label
.
Subversion: Selective and Incremental Conversions
-
The Subversion revision ranges can be limited using the configuration options
startRevision
andendRevision
. The default is to start at revision 1 and import all revisions, defined by settingendRevision
to 0.com.p4convert.svn.start=1 com.p4convert.svn.end=0
-
Incremental conversions are possible using the Import Mode. It is important to take note of the last imported Subversion revision and start from the next revision for the subsequent import. For example: to import the first 100 subversion revisions, set the config options to:
com.p4convert.svn.start=1 com.p4convert.svn.end=100
-
At the start of the import the following summary is displayed: (In this example only the first 100 revisions are imported out of 23000.)
Importing Subversion ranges: start: 1 end: 100 last: 23000
-
For the next incremental import (say another 100 revisions) set the revision ranges to...
com.p4convert.svn.start=101 com.p4convert.svn.end=200
-
The start and end revisions can be override on the command line using the
--start
and--end
flags. For example, importing just Subversion revision 201.java -jar convert.jar --config=default.cfg --start=201 --end=201
-
In adition to overriding the start and end revisions you can override Subversion dump file location using the
--repo
flags. For example, a range of revisions from an incremental dump.svnadmin dump /path/to/repo/ -r 202:300 --incremental > 202-300.dump java -jar convert.jar --config=default.cfg --repo=202-300.dump --start=202 --end=300
Filtering Subversion paths
The default behavior is not to exclude any path in Subversion; however for
large repositories with many tags/
folders or
situations where only part of a Subversion repository is to be converted
you may wish to exclude certain Subversion paths. Subversion path
exclusion is possible using two map files exclude.map
and include.map
.
The filtering is based on matching the Subversion path to a regular
expression in the map files. The exclude.map
file is
processed first and if the pattern matches part of the path then that path
is skipped. The include.map
file can be used to
overlay the exclude.map
file re-adding paths that
were skipped. Before a conversion can start the filters must be verified
against the Dump file (typically this is fairly quick and a displays a
progress indicator).
The verification step is to prevent the situation where an excluded path
is relied on at a later point in the history for a branch, copy or merge
action. If such a situation is found the paths are logged and the excluded
source path
is added to the
issue.map
file. To resolve the issue the exclusion
should be removed from the exclude.map
file or a
regular expression, based on the issue.map
file,
added to the include.map
file.
For example; to exclude all Subversion tags in the folder 'tags/', create
an exclusion map file exclude.map
:
# exclude Subversion tags:
^tags/.*
Then start the conversion to verify the map:
pallen-mac:main$ java -jar dist/p4convert.jar --config=Config/foo.cfg loading ChangeMap: changeMap.txt loading TypeMap: types.map importing revisions: 1 to 20635 out of 20635 exclude.map: ^tags/.* Verifying exclusion map... issue: tags/rel-1.0.14/api issue: tags/rel-1.0.14/sys issue: tags/rel-2.0.3 Issues found, saving issue map... Caught EXIT shutting down ...
Looking at the reported issues the tags 'rel-1.0.14' and 'rel-2.0.3' have some actions that conflict with our exclusion, to resolve this simply add the exclusions to the 'include.map' file:
# issues reported for tags/ folder
^tags/rel-1.0.14/.*
^tags/rel-2.0.3/.*
Labeling Subversion Paths
Whilst Subversion does not have a specific label entity may users follow the 'tags' convention. If a 'tag' path is pure (in that it has not had subsequent changes to the content) it can be converted to a Perforce label.
-
To enable labeling of Subversion tags, set the
com.p4convert.svn.labels
totrue
and define maps:exclude.map
andinclude.map
.com.p4convert.svn.labels=true
-
The maps are designed with resect to branches. The default behaviour is for all Subversion tag operations to be imported as Perforce branches. In order to change the behaviour and attempt to import the Subversion tag as a Perforce label the path must be in the exclude map.
For example; to import all Subversion tags in the folder 'tags/' as Perforce labels, create an exclusion map file
exclude.map
:# exclude Subversion tags: ^tags/.*
-
A Perforce label will need a unique name space derived from the 'tags' path. The conversion tool provides a depth and regular expression configuration option to assist.
com.p4convert.svn.labelFormat=svn_label:{depth}
com.p4convert.svn.labelDepth=2
The depth determins how much of the path to use for the unique label name. Typically a value of '2' for convertional Subversion usage. With a depth of '2' the first two elements of the 'tags' path are then held in an array and used by a regular expression to generate the label name.
For example, given a Subversion tag located
tags/1.0.0/
the following regular expressions provide the coresponding label names:com.p4convert.svn.labelFormat=label:{2} label:1.0.0 com.p4convert.svn.labelFormat={1}-{2} tags-1.0.0
Note
An empty expression will use the original path.
com.p4convert.svn.labelFormat= tags/1.0.0/
Note
If a keyword of
{depth}
is used it will be substituted with it's value.com.p4convert.svn.labelFormat=label_{depth} label_1.0.0
Changelist Offset options
-
The default offset is
0
(i.e. not offsetting). Offset is currently available for Convert Mode and allows subversion revisions to be converted into Perforce changelists by a fixed offset:com.p4convert.p4.offset=0
-
After the conversion a changeMap file is appended locally containing a Subversion revision number to Perforce change-list mapping. The file name is configured by the option:
com.p4convert.log.changeMap=changeMap.txt
A changemap looks like this:
# <Change>, <SVN revision> 1, 1 2, 2 3, 3 ...
Unicode Support
The following Unicode enable options apply to Unicode support for Import Mode and Convert Mode. The Charset options are only applicable to Import Mode, when translating a file through the Perforce client. In Convert Mode archive files are always written in UTF-8 for a Unicode enabled Perforce server.
Defaults (for non-Unicode servers):
com.p4convert.p4.unicode=false
com.p4convert.p4.translate=true
com.p4convert.p4.charset=<none>
In some situations it is preferable to import text files as-is (untranslated).
Typically this is true for a non-unicode environment where all the user are on
Windows clients. To disable translation set the following option to
false
.
com.p4convert.p4.translate=false
When translation is disabled high-ascii text uses the content type
TEXT-RAW
and the following warning is disabled:
... Non-unicode server, downgrading file to text
Recommended configuration for a Unicode conversion:
com.p4convert.p4.unicode=true
com.p4convert.p4.translate=true
com.p4convert.p4.charset=utf8
For Unicode conversions set the JVM arg:
-Dfile.encoding=UTF-8
Some Solaris and Linux conversions may need the locale set:
export LC_ALL=en_GB.UTF-8
Once a Perforce server is switched to Unicode enabled mode
(-xi
), all client workspaces need to define a
character set. For details see:
http://answers.perforce.com/articles/KB_Article/Internationalization-and-Localization
Note
A non-Unicode enabled Perforce Server will accept UTF16 files.
Normalisation
Platform Unicode normalisation is detected when the configuration file
is generated, however it can be changed by setting the following
configuration option to NFC
or
NFD
:
com.p4convert.p4.normalisation=NFD
The default detection is based on the following:
Platform |
Normalization |
---|---|
Windows |
|
Mac |
|
*nix/*nux |
|
Sun |
|
Subversion Properties
By default, the converter parses Subversion properties as UTF-8 strings.
The conversion uses properties such as svn:log
,
svn:author
for attributes in Perforce and must decode
the byte sequence to UTF-8. In some data sets Windows users may have
added high ASCII characters in one or more code pages. This release now
supports a configuration option:
com.p4convert.svn.propTextType=UNKNOWN
The following strings denote the supported char-sets:
Big5 | IBM424_rtl | ISO-8859-7 | UTF-16LE |
BINARY | ISO-2022-CN | ISO-8859-8 | UTF-32BE |
EUC-JP | ISO-2022-JP | ISO-8859-9 | UTF-32LE |
EUC-KR | ISO-2022-KR | KOI8-R | UTF-8 |
GB18030 | ISO-8859-1 | Shift_JIS | windows-1251 |
IBM420_ltr | ISO-8859-2 | UNKNOWN | windows-1252 |
IBM420_rtl | ISO-8859-5 | US-ASCII | windows-1254 |
IBM424_ltr | ISO-8859-6 | UTF-16BE | windows-1256 |
The first scan is always UTF-8
followed by the
configuration option. BINARY
implies a skip and the
string <binary property>
is inserted.
Advanced Configuration
Directory Properties
The following options allow Subversion Directory Properties to be stored
as versioned files in Perforce. To enable this mode set the following
property to true
:
com.p4convert.svn.propEnabled=true
To select the property name and format: (Note: only
ini
mode is supported)
com.p4convert.svn.propEncoding=ini
com.p4convert.svn.propName=.svn.properties
Empty changelists
The following property will attempt to skip empty changes (where the change contains no revisions). This is typically the default behavior of the client or Import Mode, so it is only really used in Convert Mode.
com.p4convert.p4.skipEmpty=false
Username translation
A username map file (users.map
) can be generated
using the --users
option and then the
right-hand-side modified with the new user name. The rename will only
occur if the conversion tool finds the users.map
file in the current working directory.
Username mapping is not currently supported for CVS conversions.
Binary file detection
Binary files can be identified by adding their extensions to the type
map file types.map
. The format is based on Perforce
typemap spec, however it is limited to paths of the form
//...
(where
.ext
.ext
is the binary extension).
Default Type map (types.map
):
TypeMap: binary //....zip binary //....gif binary //....png binary //....jpg binary //....dll binary //....class binary //....jar binary //....ecsfr
Modification bits (+mxwlk
) are supported and can be
added using the type mapping. Binary detection though the type map is
recommended as conversion is much faster. Binary files not identified in
the type map will be scanned by the ICU4J libraries and if no
text/Unicode match is found they are assumed to be binary.
Warning
ICU4J may incorrectly identify small binary files as text creating sync issues on Windows clients.
Changelist Description Format
The logRevID
option can be used to reformat the
Subversion revision descriptions to include the revision ID using the
template:
-
<rev>
substituted with the Subversion revision -
<description>
substituted with the Subversion log
The default value (as-is):
com.p4convert.p4.logRevID=<description>
Case Sensitivity
The platform case sensitivity is detected when generating the configuration file. There is normally no reason to change this behavior from the detected defaults. Conversions between different platforms should be avoided especially when converting from a case sensitive environment (such as Linux) to a case insensitive environment (such as Windows). The advanced case handling options supported are set using one of the following options:
com.p4convert.p4.caseMode=FIRST
-
NONE
- treat all paths as case sensitive (Linux). -
LOWER
- convert all paths to lower case -
UPPER
- convert all paths to upper case -
FIRST
- use the first encountered case combination (Windows)
When using Convert Mode the generated Perforce archive files are
based on the platform's case sensitivity. However on Linux platforms it
can be useful to store archive files as if on a case-insensitive server
(p4d -C1). This can be simulated by setting the following
option to true
:
com.p4convert.p4.lowerCase=true
Important
If this option is set the path to the Perforce root directory, defined
by com.p4convert.adv.p4root
, must be in
lower case and the case mode of
FIRST
must be used.
RCS Keyword expansion (svn:keywords)
By default, RCS keyword expansion attributes are imported; however setting the configuration option:
com.p4convert.svn.keepKeyword=true
Setting a value of false
will ignore all previous
keyword attributes and import the files as normal text. See
keyword notes for
known issues.
Merge Information (svn:mergeinfo)
Supports Subversion 1.5-1.7 merge information and calculates the corresponding Perforce integration credit for the various actions. The feature is not enabled by default and if required the following configuration option must be set to true:
com.p4convert.svn.mergeInfoEnabled=true