# P4Convert: User Guide (April 2015)

# Configuration

The java conversion requires a configuration file to setup the necessary options. A default configuration file `default.cfg` can be generated by running the converter with the `--default` parameter and specifying `CVS` or `SVN`.

For example: to generate a default configuration file for CVS, run...

    java -jar p4convert.jar --type=CVS --default

Once you have `default.cfg`, copy it to `yourconfig.cfg` and then customize it as needed for the conversion.

## Core converter settings.

The following settings appear at the top of the configuration file and are automatically defined at the time of generation. The Core converter settings should not be modified.

1. Configuration file schema version. Used to identify older configuration formats.

       com.p4convert.core.schema=5.50

2. SCM conversion type. Either set SVN or CVS configuration options.

       com.p4convert.core.scmType=SVN

3. Reserved for testing framework. Must be set to `false` for normal operation.

       com.p4convert.core.test=false

4. Conversion tool's release version.

       com.p4convert.core.version=PUBLIC.10690

## General Perforce converter settings.

1. Set the conversion mode to `IMPORT` for Import Mode and to `CONVERT` for Conversion Mode.

       com.p4convert.p4.mode=IMPORT

2. Import depot path and depot to be created, e.g. `//`import`/...`

       com.p4convert.core.depotPath=import

3. An optional sub path can be specified below the depot, e.g. `//import/`sub`/...` (Note that the path must end with a slash (`/`) even if no path is used). The default value is `/`.

       com.p4convert.core.subPath=sub/

## General Subversion converter settings.

If the configuration file has been correctly generated using the `--type=SVN --default` the Subversion configuration options will appear as a block using the `com.p4convert.svn` name space. These settings will only apply if the SCM mode is set to `SVN`:

    com.p4convert.core.scmType=SVN

1. Local path to Subversion dump file.

       com.p4convert.svn.dumpFile=/Users/bruno/MyDumpFile.dmp

## General CVS converter settings.

If the configuration file has been correctly generated using the `--type=CVS --default` the CVS configuration options will appear as a block using the `com.p4convert.cvs` name space. These settings will only apply if the SCM mode is set to `CVS`:

    com.p4convert.core.scmType=CVS

1. Local path to CVSROOT files. (A deeper path is permitted to limit the scope of the import).

       com.p4convert.cvs.cvsroot=/Users/bruno/CVSROOT/

2. (Optionally) specify a CVS module to import from within the CVSROOT.

       com.p4convert.cvs.cvsmodule=projX

3. CVS time window. When no `commitid` is defined in the RCS file, the converter will calculate which revisions should be grouped into a changelist. The time window specifies the maximum range the converter should look ahead for matching revisions. (milliseconds)

       com.p4convert.cvs.timeWindow=20000

4. CVS temporary directory. All the CVS assets are reformatted into full text files. The specified temporary directory is used to store the content during conversion.

       com.p4convert.cvs.tmpDir=tmp

## Mapping CVS Paths

CVS paths are mapped into Perforce using the original RCS file structure, but prefixed with the branch name. The converter uses a branch name 'main' for the initial 1.1 branch and the symbol names for subsequent branches.

If requires an optional path map can be provided to remap the CVS path to a new structure within Perforce. When designing a path map, care must be taken to insure that all CVS paths have a destination mapping.

1. To enable CVS path mapping create a file called `path.map` with regex and group match.

2. Only the first matching entry is used.

3. The regex and group match are seperated by ', ' (or in regex terms ',\s+').

4. Lines starting with '#' are ignored.

For example, 'trunk' is renamed to 'main', but other entries are left as-is.

    # path.map
    trunk/(.*), //import/main/{1}
    (.*), //import/{1}

Note: if no file is found the default 'depot' and 'subPath' options are used to generate the map, preserving the original behaviour. CVS paths will always stat with the 'branch' name. 'main' for 1.1 and the symbol for other branches.

    # path.map
    main/projA/(.*), //import/projA/MAIN/{1}
    release_(.*)/projA/(.*), //import/projA/REL{1}/{2}
    (.*)/projA/(.*), //import/projA/TAG-{1}/{2}
    (.*), //import/unexpected/{1}

### Note

Adding a catch all is a good idea (as shown in the last line with the 'unexpected' directory).

## Labeling CVS Paths

CVS will tag revisions with symbols, some of these symbols may refer to labels, whilst others are branch names. If labeling is disabled (default), then only symbols with history are converted to Perforce branches, the remaining symbols are ignored. However, if labels are enabled then the remaining symbols from the branch detection are converted to Perforce Labels.

1. To enable labeling of CVS symbols tags, set the `com.p4convert.cvs.labels` to `true`.

2. In multiple CVS conversions there is the potential for namespace clashing if the label has been previously used. By default, the converter will use the CVS symbol as the name for the Perforce label. However, this can be formatted by setting the following option: `com.p4convert.cvs.labelFormat`. The parameter `{symbol}` will be substituted by the CVS symbol name.

   For example:

       com.p4convert.cvs.labels=true
       com.p4convert.cvs.labelFormat=import_{symbol}_label

   Will import a CVS symbol called `REL1.0` as a Perforce label `import_REL1.0_label`.

## Subversion: Selective and Incremental Conversions

1. The Subversion revision ranges can be limited using the configuration options `startRevision` and `endRevision`. The default is to start at revision 1 and import all revisions, defined by setting `endRevision` to 0.

       com.p4convert.svn.start=1
       com.p4convert.svn.end=0

2. Incremental conversions are possible using the Import Mode. It is important to take note of the last imported Subversion revision and start from the next revision for the subsequent import. For example: to import the first 100 subversion revisions, set the config options to:

       com.p4convert.svn.start=1
       com.p4convert.svn.end=100

3. At the start of the import the following summary is displayed: (In this example only the first 100 revisions are imported out of 23000.)

       Importing Subversion ranges:
         start: 1
         end:   100
         last:  23000

4. For the next incremental import (say another 100 revisions) set the revision ranges to...

       com.p4convert.svn.start=101
       com.p4convert.svn.end=200

5. The start and end revisions can be override on the command line using the `--start` and `--end` flags. For example, importing just Subversion revision 201.

       java -jar convert.jar --config=default.cfg --start=201 --end=201

6. In adition to overriding the start and end revisions you can override Subversion dump file location using the `--repo` flags. For example, a range of revisions from an incremental dump.

       svnadmin dump /path/to/repo/ -r 202:300 --incremental > 202-300.dump

       java -jar convert.jar --config=default.cfg --repo=202-300.dump --start=202 --end=300

## Filtering Subversion paths

The default behavior is not to exclude any path in Subversion; however for large repositories with many `tags/` folders or situations where only part of a Subversion repository is to be converted you may wish to exclude certain Subversion paths. Subversion path exclusion is possible using two map files `exclude.map` and `include.map`.

The filtering is based on matching the Subversion path to a regular expression in the map files. The `exclude.map` file is processed first and if the pattern matches part of the path then that path is skipped. The `include.map` file can be used to overlay the `exclude.map` file re-adding paths that were skipped. Before a conversion can start the filters must be verified against the Dump file (typically this is fairly quick and a displays a progress indicator).

The verification step is to prevent the situation where an excluded path is relied on at a later point in the history for a branch, copy or merge action. If such a situation is found the paths are logged and the excluded `source path` is added to the `issue.map` file. To resolve the issue the exclusion should be removed from the `exclude.map` file or a regular expression, based on the `issue.map` file, added to the `include.map` file.

For example; to exclude all Subversion tags in the folder 'tags/', create an exclusion map file `exclude.map`:

    # exclude Subversion tags:
    ^tags/.*

Then start the conversion to verify the map:

    pallen-mac:main$ java -jar dist/p4convert.jar --config=Config/foo.cfg
    loading ChangeMap: 	changeMap.txt
    loading TypeMap: 	types.map
    importing revisions: 	1 to 20635 out of 20635
    exclude.map: ^tags/.*
    Verifying exclusion map...
    issue: tags/rel-1.0.14/api
    issue: tags/rel-1.0.14/sys
    issue: tags/rel-2.0.3
    Issues found, saving issue map...
    Caught EXIT shutting down ...

Looking at the reported issues the tags 'rel-1.0.14' and 'rel-2.0.3' have some actions that conflict with our exclusion, to resolve this simply add the exclusions to the 'include.map' file:

    # issues reported for tags/ folder
    ^tags/rel-1.0.14/.*
    ^tags/rel-2.0.3/.*

## Labeling Subversion Paths

Whilst Subversion does not have a specific label entity may users follow the 'tags' convention. If a 'tag' path is pure (in that it has not had subsequent changes to the content) it can be converted to a Perforce label.

1. To enable labeling of Subversion tags, set the `com.p4convert.svn.labels` to `true` and define maps: `exclude.map` and `include.map`.

       com.p4convert.svn.labels=true

2. The maps are designed with resect to branches. The default behaviour is for all Subversion tag operations to be imported as Perforce branches. In order to change the behaviour and attempt to import the Subversion tag as a Perforce label the path must be in the exclude map.

   For example; to import all Subversion tags in the folder 'tags/' as Perforce labels, create an exclusion map file `exclude.map`:

       # exclude Subversion tags:
       ^tags/.*

3. A Perforce label will need a unique name space derived from the 'tags' path. The conversion tool provides a depth and regular expression configuration option to assist.

       com.p4convert.svn.labelFormat=svn_label:{depth}

       com.p4convert.svn.labelDepth=2

   The depth determins how much of the path to use for the unique label name. Typically a value of '2' for convertional Subversion usage. With a depth of '2' the first two elements of the 'tags' path are then held in an array and used by a regular expression to generate the label name.

   For example, given a Subversion tag located `tags/1.0.0/` the following regular expressions provide the coresponding label names:

       com.p4convert.svn.labelFormat=label:{2}         label:1.0.0
       com.p4convert.svn.labelFormat={1}-{2}           tags-1.0.0

   ### Note

   An empty expression will use the original path.

       com.p4convert.svn.labelFormat=                  tags/1.0.0/

   ### Note

   If a keyword of `{depth}` is used it will be substituted with it's value.

       com.p4convert.svn.labelFormat=label_{depth}     label_1.0.0

## Changelist Offset options

1. The default offset is `0` (i.e. not offsetting). Offset is currently available for Convert Mode and allows subversion revisions to be converted into Perforce changelists by a fixed offset:

       com.p4convert.p4.offset=0

2. After the conversion a changeMap file is appended locally containing a Subversion revision number to Perforce change-list mapping. The file name is configured by the option:

       com.p4convert.log.changeMap=changeMap.txt

   A changemap looks like this:

       # <Change>, <SVN revision>
       1, 1
       2, 2
       3, 3
       ...

## Unicode Support

The following Unicode enable options apply to Unicode support for Import Mode and Convert Mode. The Charset options are only applicable to Import Mode, when translating a file through the Perforce client. In Convert Mode archive files are always written in UTF-8 for a Unicode enabled Perforce server.

Defaults (for non-Unicode servers):

    com.p4convert.p4.unicode=false
    com.p4convert.p4.translate=true
    com.p4convert.p4.charset=<none>

In some situations it is preferable to import text files as-is (untranslated). Typically this is true for a non-unicode environment where all the user are on Windows clients. To disable translation set the following option to `false`.

    com.p4convert.p4.translate=false

When translation is disabled high-ascii text uses the content type `TEXT-RAW` and the following warning is disabled:

    ... Non-unicode server, downgrading file to text

Recommended configuration for a Unicode conversion:

    com.p4convert.p4.unicode=true
    com.p4convert.p4.translate=true
    com.p4convert.p4.charset=utf8

For Unicode conversions set the JVM arg:

    -Dfile.encoding=UTF-8

Some Solaris and Linux conversions may need the locale set:

    export LC_ALL=en_GB.UTF-8

Once a Perforce server is switched to Unicode enabled mode (`-xi`), all client workspaces need to define a character set. For details see:

http://answers.perforce.com/articles/KB_Article/Internationalization-and-Localization

### Note

A non-Unicode enabled Perforce Server will accept UTF16 files.

### Normalisation

Platform Unicode normalisation is detected when the configuration file is generated, however it can be changed by setting the following configuration option to `NFC` or `NFD`:

    com.p4convert.p4.normalisation=NFD

The default detection is based on the following:

Platform | Normalization
--- | ---
Windows | `NFC`
Mac | `NFD`
*nix/*nux | `NFC`
Sun | `NFC`

### Subversion Properties

By default, the converter parses Subversion properties as UTF-8 strings. The conversion uses properties such as `svn:log`, `svn:author` for attributes in Perforce and must decode the byte sequence to UTF-8. In some data sets Windows users may have added high ASCII characters in one or more code pages. This release now supports a configuration option:

    com.p4convert.svn.propTextType=UNKNOWN

The following strings denote the supported char-sets:

Big5 | IBM424_rtl | ISO-8859-7 | UTF-16LE
BINARY | ISO-2022-CN | ISO-8859-8 | UTF-32BE
EUC-JP | ISO-2022-JP | ISO-8859-9 | UTF-32LE
EUC-KR | ISO-2022-KR | KOI8-R | UTF-8
GB18030 | ISO-8859-1 | Shift_JIS | windows-1251
IBM420_ltr | ISO-8859-2 | UNKNOWN | windows-1252
IBM420_rtl | ISO-8859-5 | US-ASCII | windows-1254
IBM424_ltr | ISO-8859-6 | UTF-16BE | windows-1256

The first scan is always `UTF-8` followed by the configuration option. `BINARY` implies a skip and the string `<binary property>` is inserted.

## Advanced Configuration

### Directory Properties

The following options allow Subversion Directory Properties to be stored as versioned files in Perforce. To enable this mode set the following property to `true`:

    com.p4convert.svn.propEnabled=true

To select the property name and format: (Note: only `ini` mode is supported)

    com.p4convert.svn.propEncoding=ini
    com.p4convert.svn.propName=.svn.properties

### Empty changelists

The following property will attempt to skip empty changes (where the change contains no revisions). This is typically the default behavior of the client or Import Mode, so it is only really used in Convert Mode.

    com.p4convert.p4.skipEmpty=false

### Username translation

A username map file (`users.map`) can be generated using the `--users` option and then the right-hand-side modified with the new user name. The rename will only occur if the conversion tool finds the `users.map` file in the current working directory. Username mapping is not currently supported for CVS conversions.

### Binary file detection

Binary files can be identified by adding their extensions to the type map file `types.map`. The format is based on Perforce typemap spec, however it is limited to paths of the form `//...`.ext`` (where `.ext` is the binary extension).

Default Type map (`types.map`):

    TypeMap:
            binary	//....zip
            binary	//....gif
            binary	//....png
            binary	//....jpg
            binary  //....dll
            binary  //....class
            binary  //....jar
            binary  //....ecsfr

Modification bits (`+mxwlk`) are supported and can be added using the type mapping. Binary detection though the type map is recommended as conversion is much faster. Binary files not identified in the type map will be scanned by the ICU4J libraries and if no text/Unicode match is found they are assumed to be binary.

### Warning

ICU4J may incorrectly identify small binary files as text creating sync issues on Windows clients.

### Changelist Description Format

The `logRevID` option can be used to reformat the Subversion revision descriptions to include the revision ID using the template:

- `<rev>` substituted with the Subversion revision

- `<description>` substituted with the Subversion log

The default value (as-is):

    com.p4convert.p4.logRevID=<description>

### Case Sensitivity

The platform case sensitivity is detected when generating the configuration file. There is normally no reason to change this behavior from the detected defaults. Conversions between different platforms should be avoided especially when converting from a case sensitive environment (such as Linux) to a case insensitive environment (such as Windows). The advanced case handling options supported are set using one of the following options:

    com.p4convert.p4.caseMode=FIRST

- `NONE` - treat all paths as case sensitive (Linux).

- `LOWER` - convert all paths to lower case

- `UPPER` - convert all paths to upper case

- `FIRST` - use the first encountered case combination (Windows)

When using Convert Mode the generated Perforce archive files are based on the platform's case sensitivity. However on Linux platforms it can be useful to store archive files as if on a case-insensitive server (p4d -C1). This can be simulated by setting the following option to `true`:

    com.p4convert.p4.lowerCase=true

### Important

If this option is set the path to the Perforce root directory, defined by `com.p4convert.adv.p4root`, must be in lower case and the case mode of `FIRST` must be used.

### RCS Keyword expansion (svn:keywords)

By default, RCS keyword expansion attributes are imported; however setting the configuration option:

    com.p4convert.svn.keepKeyword=true

Setting a value of `false` will ignore all previous keyword attributes and import the files as normal text. See keyword notes for known issues.

### Merge Information (svn:mergeinfo)

Supports Subversion 1.5-1.7 merge information and calculates the corresponding Perforce integration credit for the various actions. The feature is not enabled by default and if required the following configuration option must be set to true:

    com.p4convert.svn.mergeInfoEnabled=true

Hide partial matches
Highlight matches
0 matching pages