Configuration
The java conversion requires a configuration file to setup the necessary
options. A default configuration file default.cfg can
be generated by running the converter with the --default
parameter and specifying CVS or SVN.
For example: to generate a default configuration file for CVS, run...
java -jar p4convert.jar --type=CVS --default
Once you have default.cfg, copy it to
yourconfig.cfg and then customize it as needed for
the conversion.
Core converter settings.
The following settings appear at the top of the configuration file
and are automatically defined at the time of generation. The
Core converter settings should not be modified.
Configuration file schema version. Used to identify older configuration
formats.
com.p4convert.core.schema=5.50
SCM conversion type. Either set SVN or CVS configuration options.
com.p4convert.core.scmType=SVN
Reserved for testing framework. Must be set to false
for normal operation.
com.p4convert.core.test=false
Conversion tool's release version.
com.p4convert.core.version=PUBLIC.10690
General Perforce converter settings.
Set the conversion mode to IMPORT for Import Mode
and to CONVERT for Conversion Mode.
com.p4convert.p4.mode=IMPORT
Import depot path and depot to be created, e.g.
//import/...
com.p4convert.core.depotPath=import
An optional sub path can be specified below the depot, e.g.
//import/sub/... (Note
that the path must end with a slash (/) even if
no path is used). The default value is /.
com.p4convert.core.subPath=sub/
General Subversion converter settings.
If the configuration file has been correctly generated using the
--type=SVN --default the Subversion configuration
options will appear as a block using the com.p4convert.svn
name space. These settings will only apply if the SCM
mode is set to SVN:
com.p4convert.core.scmType=SVN
Local path to Subversion dump file.
com.p4convert.svn.dumpFile=/Users/bruno/MyDumpFile.dmp
General CVS converter settings.
If the configuration file has been correctly generated using the
--type=CVS --default the CVS configuration
options will appear as a block using the com.p4convert.cvs
name space. These settings will only apply if the SCM
mode is set to CVS:
com.p4convert.core.scmType=CVS
Local path to CVSROOT files. (A deeper path is permitted to limit
the scope of the import).
com.p4convert.cvs.cvsroot=/Users/bruno/CVSROOT/
(Optionally) specify a CVS module to import from within the CVSROOT.
com.p4convert.cvs.cvsmodule=projX
CVS time window. When no commitid is
defined in the RCS file, the converter will calculate which
revisions should be grouped into a changelist. The time window
specifies the maximum range the converter should look ahead for
matching revisions. (milliseconds)
com.p4convert.cvs.timeWindow=20000
CVS temporary directory. All the CVS assets are reformatted into
full text files. The specified temporary directory is used to store
the content during conversion.
com.p4convert.cvs.tmpDir=tmpLabeling CVS Paths
CVS will tag revisions with symbols, some of these symbols may refer to
labels, whilst others are branch names. If labeling is disabled (default),
then only symbols with history are converted to Perforce branches, the
remaining symbols are ignored. However, if labels are enabled then the
remaining symbols from the branch detection are converted to Perforce Labels.
To enable labeling of CVS symbols tags, set the
com.p4convert.cvs.labels to
true.
In multiple CVS conversions there is the potential for namespace clashing
if the label has been previously used. By default, the converter will use
the CVS symbol as the name for the Perforce label. However, this can be
formatted by setting the following option:
com.p4convert.cvs.labelFormat. The parameter
{symbol} will be substituted by the CVS symbol name.
For example:
com.p4convert.cvs.labels=true
com.p4convert.cvs.labelFormat=import_{symbol}_label
Will import a CVS symbol called REL1.0 as a Perforce
label import_REL1.0_label.
Subversion: Selective and Incremental Conversions
The Subversion revision ranges can be limited using the configuration
options startRevision and
endRevision. The default is to start at
revision 1 and import all revisions, defined by setting
endRevision to 0.
com.p4convert.svn.start=1
com.p4convert.svn.end=0
Incremental conversions are possible using the Import Mode. It is
important to take note of the last imported Subversion revision and
start from the next revision for the subsequent import. For example:
to import the first 100 subversion revisions, set the config options
to:
com.p4convert.svn.start=1
com.p4convert.svn.end=100
At the start of the import the following summary is displayed: (In
this example only the first 100 revisions are imported out of 23000.)
Importing Subversion ranges:
start: 1
end: 100
last: 23000
For the next incremental import (say another 100 revisions) set the
revision ranges to...
com.p4convert.svn.start=101
com.p4convert.svn.end=200
The start and end revisions can be override on the command line
using the --start and
--end flags. For example, importing just
Subversion revision 201.
java -jar convert.jar --config=default.cfg --start=201 --end=201
In adition to overriding the start and end revisions you can
override Subversion dump file location using the
--repo flags. For example, a range of
revisions from an incremental dump.
svnadmin dump /path/to/repo/ -r 202:300 --incremental > 202-300.dump
java -jar convert.jar --config=default.cfg --repo=202-300.dump --start=202 --end=300
Filtering Subversion paths
The default behavior is not to exclude any path in Subversion; however for
large repositories with many tags/ folders or
situations where only part of a Subversion repository is to be converted
you may wish to exclude certain Subversion paths. Subversion path
exclusion is possible using two map files exclude.map
and include.map.
The filtering is based on matching the Subversion path to a regular
expression in the map files. The exclude.map file is
processed first and if the pattern matches part of the path then that path
is skipped. The include.map file can be used to
overlay the exclude.map file re-adding paths that
were skipped. Before a conversion can start the filters must be verified
against the Dump file (typically this is fairly quick and a displays a
progress indicator).
The verification step is to prevent the situation where an excluded path
is relied on at a later point in the history for a branch, copy or merge
action. If such a situation is found the paths are logged and the excluded
source path is added to the
issue.map file. To resolve the issue the exclusion
should be removed from the exclude.map file or a
regular expression, based on the issue.map file,
added to the include.map file.
For example; to exclude all Subversion tags in the folder 'tags/', create
an exclusion map file exclude.map:
# exclude Subversion tags:
^tags/.*
Then start the conversion to verify the map:
pallen-mac:main$ java -jar dist/p4convert.jar --config=Config/foo.cfg
loading ChangeMap: changeMap.txt
loading TypeMap: types.map
importing revisions: 1 to 20635 out of 20635
exclude.map: ^tags/.*
Verifying exclusion map...
issue: tags/rel-1.0.14/api
issue: tags/rel-1.0.14/sys
issue: tags/rel-2.0.3
Issues found, saving issue map...
Caught EXIT shutting down ...
Looking at the reported issues the tags 'rel-1.0.14' and 'rel-2.0.3' have
some actions that conflict with our exclusion, to resolve this simply add
the exclusions to the 'include.map' file:
# issues reported for tags/ folder
^tags/rel-1.0.14/.*
^tags/rel-2.0.3/.*
Labeling Subversion Paths
Whilst Subversion does not have a specific label entity may users follow
the 'tags' convention. If a 'tag' path is pure (in that it has not had
subsequent changes to the content) it can be converted to a Perforce
label.
To enable labeling of Subversion tags, set the
com.p4convert.svn.labels to
true and define maps:
exclude.map and include.map.
com.p4convert.svn.labels=true
The maps are designed with resect to branches. The default behaviour is
for all Subversion tag operations to be imported as Perforce branches.
In order to change the behaviour and attempt to import the Subversion tag
as a Perforce label the path must be in the exclude map.
For example; to import all Subversion tags in the folder 'tags/' as
Perforce labels, create an exclusion map file
exclude.map:
# exclude Subversion tags:
^tags/.*
A Perforce label will need a unique name space derived from the 'tags'
path. The conversion tool provides a depth and regular expression
configuration option to assist.
com.p4convert.svn.labelFormat=svn_label:{depth}
com.p4convert.svn.labelDepth=2
The depth determins how much of the path to use for the unique label
name. Typically a value of '2' for convertional Subversion usage.
With a depth of '2' the first two elements of the 'tags' path are then
held in an array and used by a regular expression to generate the label
name.
For example, given a Subversion tag located tags/1.0.0/
the following regular expressions provide the coresponding label names:
com.p4convert.svn.labelFormat=label:{2} label:1.0.0
com.p4convert.svn.labelFormat={1}-{2} tags-1.0.0
An empty expression will use the original path.
com.p4convert.svn.labelFormat= tags/1.0.0/
If a keyword of {depth} is used it will be
substituted with it's value.
com.p4convert.svn.labelFormat=label_{depth} label_1.0.0
Changelist Offset options
The default offset is 0 (i.e. not offsetting).
Offset is currently available for Convert Mode and allows subversion
revisions to be converted into Perforce changelists by a fixed offset:
com.p4convert.p4.offset=0
After the conversion a changeMap file is appended
locally containing a Subversion revision number to Perforce
change-list mapping. The file name is configured by the option:
com.p4convert.log.changeMap=changeMap.txt
A changemap looks like this:
# <Change>, <SVN revision>
1, 1
2, 2
3, 3
...
Unicode Support
The following Unicode enable options apply to Unicode support for Import
Mode and Convert Mode. The Charset options are only applicable to Import
Mode, when translating a file through the Perforce client. In Convert Mode
archive files are always written in UTF-8 for a Unicode enabled Perforce
server.
Defaults (for non-Unicode servers):
com.p4convert.p4.unicode=false
com.p4convert.p4.translate=true
com.p4convert.p4.charset=<none>
In some situations it is preferable to import text files as-is (untranslated).
Typically this is true for a non-unicode environment where all the user are on
Windows clients. To disable translation set the following option to
false.
com.p4convert.p4.translate=false
When translation is disabled high-ascii text uses the content type
TEXT-RAW and the following warning is disabled:
... Non-unicode server, downgrading file to text
Recommended configuration for a Unicode conversion:
com.p4convert.p4.unicode=true
com.p4convert.p4.translate=true
com.p4convert.p4.charset=utf8
For Unicode conversions set the JVM arg:
-Dfile.encoding=UTF-8
Some Solaris and Linux conversions may need the locale set:
export LC_ALL=en_GB.UTF-8
Once a Perforce server is switched to Unicode enabled mode
(-xi), all client workspaces need to define a
character set. For details see:
http://answers.perforce.com/articles/KB_Article/Internationalization-and-Localization
A non-Unicode enabled Perforce Server will accept UTF16 files.
Normalisation
Platform Unicode normalisation is detected when the configuration file
is generated, however it can be changed by setting the following
configuration option to NFC or
NFD:
com.p4convert.p4.normalisation=NFD
The default detection is based on the following:
PlatformNormalizationWindowsNFCMacNFD*nix/*nuxNFCSunNFCSubversion Properties
By default, the converter parses Subversion properties as UTF-8 strings.
The conversion uses properties such as svn:log,
svn:author for attributes in Perforce and must decode
the byte sequence to UTF-8. In some data sets Windows users may have
added high ASCII characters in one or more code pages. This release now
supports a configuration option:
com.p4convert.svn.propTextType=UNKNOWN
The following strings denote the supported char-sets:
Big5BINARYEUC-JPEUC-KRGB18030IBM420_ltrIBM420_rtlIBM424_ltrIBM424_rtlISO-2022-CNISO-2022-JPISO-2022-KRISO-8859-1ISO-8859-2ISO-8859-5ISO-8859-6ISO-8859-7ISO-8859-8ISO-8859-9KOI8-RShift_JISUNKNOWNUS-ASCIIUTF-16BEUTF-16LEUTF-32BEUTF-32LEUTF-8windows-1251windows-1252windows-1254windows-1256
The first scan is always UTF-8 followed by the
configuration option. BINARY implies a skip and the
string <binary property> is inserted.
Advanced ConfigurationDirectory Properties
The following options allow Subversion Directory Properties to be stored
as versioned files in Perforce. To enable this mode set the following
property to true:
com.p4convert.svn.propEnabled=true
To select the property name and format: (Note: only
ini mode is supported)
com.p4convert.svn.propEncoding=ini
com.p4convert.svn.propName=.svn.properties
Empty changelists
The following property will attempt to skip empty changes (where the
change contains no revisions). This is typically the default behavior of
the client or Import Mode, so it is only really used in Convert Mode.
com.p4convert.p4.skipEmpty=false
Username translation
A username map file (users.map) can be generated
using the --users option and then the
right-hand-side modified with the new user name. The rename will only
occur if the conversion tool finds the users.map
file in the current working directory.
Username mapping is not currently supported for CVS conversions.
Binary file detection
Binary files can be identified by adding their extensions to the type
map file types.map. The format is based on Perforce
typemap spec, however it is limited to paths of the form
//....ext (where
.ext is the binary extension).
Default Type map (types.map):
TypeMap:
binary //....zip
binary //....gif
binary //....png
binary //....jpg
binary //....dll
binary //....class
binary //....jar
binary //....ecsfr
Modification bits (+mxwlk) are supported and can be
added using the type mapping. Binary detection though the type map is
recommended as conversion is much faster. Binary files not identified in
the type map will be scanned by the ICU4J libraries and if no
text/Unicode match is found they are assumed to be binary.
ICU4J may incorrectly identify small binary files as text creating
sync issues on Windows clients.
Changelist Description Format
The logRevID option can be used to reformat the
Subversion revision descriptions to include the revision ID using the
template:
<rev> substituted with the
Subversion revision
<description> substituted with the
Subversion log
The default value (as-is):
com.p4convert.p4.logRevID=<description>
Case Sensitivity
The platform case sensitivity is detected when generating the
configuration file. There is normally no reason to change this behavior
from the detected defaults. Conversions between different platforms
should be avoided especially when converting from a case sensitive
environment (such as Linux) to a case insensitive environment (such as
Windows). The advanced case handling options supported are set using one
of the following options:
com.p4convert.p4.caseMode=FIRST
NONE - treat all paths as case sensitive (Linux).
LOWER - convert all paths to lower case
UPPER - convert all paths to upper case
FIRST - use the first encountered case
combination (Windows)
When using Convert Mode the generated Perforce archive files are
based on the platform's case sensitivity. However on Linux platforms it
can be useful to store archive files as if on a case-insensitive server
(p4d -C1). This can be simulated by setting the following
option to true:
com.p4convert.p4.lowerCase=true
If this option is set the path to the Perforce root directory, defined
by com.p4convert.adv.p4root, must be in
lower case and the case mode of
FIRST must be used.
RCS Keyword expansion (svn:keywords)
By default, RCS keyword expansion attributes are imported; however
setting the configuration option:
com.p4convert.svn.keepKeyword=true
Setting a value of false will ignore all previous
keyword attributes and import the files as normal text. See
keyword notes for
known issues.
Merge Information (svn:mergeinfo)
Supports Subversion 1.5-1.7 merge information and calculates the
corresponding Perforce integration credit for the various actions. The
feature is not enabled by default and if required the following
configuration option must be set to true:
com.p4convert.svn.mergeInfoEnabled=true