SCM Tool Evaluation

Current Scenario

ECD has reached a point where it must evaluate its current SCM needs. I have identified the following problems with the current SCM, Microsoft Visual SourceSafe (VSS):

VSS does not allow cross-platform development – currently, developers who are developing in Linux have to reboot in Windows NT in order to access VSS to check in and out their changes. This is a big productivity hit and a major source of frustration.
VSS does not handle branching or merging of files. This has required the manual creation and maintenance of multiple "versioned" source trees, which are essentially snapshots of the source tree at different points in time. This creates a lot of redundancy and defeats the purpose of a source code control system. If a fix is made in one branch but also needs to be propagated to other branches, it is up to the developer to remember to do so, which is error prone. VSS also does not provide an automated merge tool. There is also no good way to track branch and merge points.
Currently there are a lot of inefficiencies because of the lack of communication, process and build automation. VSS does not facilitate build automation or the streamlining of processes.
VSS does not provide the ability to dial up from home.
VSS is known to corrupt archives from time to time. Dev just reported that if you edit a file with java extension in VSS, it actually corrupts some of the string values.
VSS does not support atomic change transactions (i.e. submitting a group of files as a single logical transaction).
VSS does not allow parallel development (for example, if a developer has one file locked, no one else can check that file out until he/she checks it back in). This is another big productivity hit.
VSS has no web interface and has very limited reporting abilities.
VSS is not true client/server (for example, the server does not know what files are on your client).
VSS is just plain slow. Yet another hit to productivity.

Requirements

The criteria listed below result from input by development, QA, production, and my impression to date of the development process at ECD.

Support for NT and Unix clients
Ease of use
Works out of a box
Easy installation and short learning curve
Client / server
Fast transactions across a thin network
Group work focused
Dial-up from home
Affordable
Lightweight
Integrated subcomponents
Minimal maintenance overhead
Minimal administration
Reliability
Flexibility

Source Code Control component

Ability to make temporal and contextual snapshots

Ability to return to a snapshot

Strong branching and merging support

Visualization of structure & history

Atomic change transactions support, that is, submitting a group of files as a single logical transaction

Build Management component

Cross-platform compilation support

Customizable

Compatible with our various development tools & third-party software

Change notification via e-mail

Support for build automation

Support for triggers and review daemons

Candidates

Microsoft Visual SourceSafe

Visual SourceSafe is included as a baseline comparison from which to analyze the other candidates.

Pro

Already in use. No conversion costs.

Con

See "Current Scenario".

Costs

Description	Quantity	License Unit Price	Total Price
Visual SourceSafe License	50	$549	$27,450

Support: Phone - $245 per incident or $1,225 per 5-pack

Web - $195 per incident or $975 per 5-pack

Training: Not available

CVS

Pro

It’s free

Provides source

Con

Not targeted at a distributed environment

Supports only a few platforms

Requires a lot of customization, and therefore resources. Does not work out of a box.

Limited functionality

Does not support atomic changes

Rational ClearCase

Pro

Powerful

Decent GUI

Strongly integrated revision/build system

Con

Tied to OS version

Very expensive

Does not support atomic changes

Supports only a few platforms

Requires fast, low-latency NFS implementation

Requires users to be connected to network to do any work. If network goes down, all builds stop.

Requires a lot of resources: full-time administrator, dedicated server, 1 full-time source management person for every 8-9 developers, lots of customization required

Costs

Talked to sales rep Drake Dunlap. Got these prices:

Description

Quantity

License Unit Price

Support Unit Price

Total Unit Price

Total
Price

ClearCase Floating License

50

$3000

$600

$3600

$180,000

*Discount:

($21,000)

Total:

$159,000

Yearly support after first year

50

$600

$30,000

* Drake explained that we get discounts of 5%, 10% then 14% at license totals of $20,000, $50,000 and $100,000 respectively. In our case our license total would be $150,000, so our discount would be $21,000.

Rational Training

"Rational University" is Rational's training solution. They appear to have a complete set of courses. The pricing is $995 per student.

Perforce

Pro

Powerful

Fast

Supports a multitude of platforms (see: http://www.perforce.com/perforce/loadprog.html#plats)

Short learning curve

Easy to use

Easy installation

Minimal administration and maintenance

Integrated defect-tracking system

Joan is an experienced user

Decent GUI

Atomic changes

Good branching model

Con

Build system is not integrated

Costs

Description

Quantity

License Unit Price

Support Unit Price

Total
Price

Perforce per-user license

50

$550

First year free

$27,500

Support/Upgrades Per Subsequent Year

50

$120

$6,000

Perforce Training

Perforce offers an 'in-depth' three-day training class at a site in San Francisco or at our site for $1200, limited to 10 attendees. I can do a basic training class in half a day to get folks up to speed.

Budget and Risk Analysis

Money

These are the price breakdown of each of the four toolsets. ClearCase will require extra hardware (servers, disks etc.). Perforce is already set up on an existing Linux machine, Madre, which is sufficient to get us started, but we should have an independent server when we go live (see Section 6). CVS will require a lot of customization and hence resources.

Vendor	Cost
Rational ClearCase	$159,000 + server cost
Perforce	$27,500 + server cost
Visual SourceSafe	$27,450
CVS	$?? – customization required

Risk

The risks of staying with the status quo are described in "Current Scenario". The main risk of going to a new SCM tool is the initial learning curve, which may initially affect productivity, but will reap long-term gains in productivity and efficiency.

Conversion

Perforce provides a set of Perl scripts (available as a ZIP file) for converting VSS archives to Perforce, preserving the original times and authors. Branches are translated verbatim; individual file revisions are grouped as much as possible into Perforce's notion of atomic changes.

Recommendation

I propose that we standardize on Perforce as our new SCM tool. The main reasons are speed - both across a thin network and operationally, price, ease of use and quality of implementation.

Training with Perforce will cost less, because I will do internal training as needed.

The SCM server should be independent, so that it operates as fast as possible and is exposed to as little risk of failure as possible. Running Madre (a Linux box) with a mirrored disk system and nightly backups would be highly reliable and fault-tolerant.