package VCP::Source::cvs ; =head1 NAME VCP::Source::cvs - A CVS repository source =head1 SYNOPSIS vcp cvs:path/... -r foo # all files in path and below labelled foo vcp cvs:path/... -r foo: # All revs of files labelled foo and newer vcp cvs:path/... -r foo: -f # All revs of files labelled foo and newer, # including files not tagged with foo. vcp cvs:path/... -r 1.1:1.10 # revs 1.1..1.10 vcp cvs:path/... -r 1.1: # revs 1.1 and up vcp cvs:path/... -D ">=2000-11-18 5:26:30" # All file revs newer than a date/time ## NOTE: Unlike cvs, vcp requires spaces after option letters. =head1 DESCRIPTION Reads a CVS repository. You must check out the files of interest in to a local working directory and run the cvs command from within that directory. This is because the "cvs log" and "cvs checkout" commands need to have a working directory to play in. This module in alpha. This doesn't deal with branches yet (at least not intentionally). That will require a bit of Deep Thought. This only deals with filespecs spelled like "/a/b/c" or "a/b/..." for now. Later, we can handle other filespecs by reading the entire log output. =head1 OPTIONS =over =item -b, --bootstrap -b '**' --bootstrap='**' -b file1[,file2[,...]] --bootstrap=file1[,file2[,...]] Forces bootstrap mode for an entire export (-b '**') or for certain files. Filenames may contain wildcards, see L<Regexp::Shellish> for details on what wildcards are accepted. For now, one thing to remember is to use '**' instead of p4's '...' wildcard. Controls how the first revision of a file is exported. A bootstrap export contains the entire contents of the first revision in the revision range. This should only be used when exporting for the first time. An incremental export contains a digest of the revision preceding the first revision in the revision range, followed by a delta record between that revision and the first revision in the range. This allows the destination import function to make sure that the incremental export begins where the last export left off. The default is decided on a per-file basis: if the first revision in the range is revision #1, the full contents are exported. Otherwise an incremental export is done for that file. This option is necessary when exporting only more recent revisions from a repository. =item --cd Used to set the CVS root directory, overriding the value of the CVSROOT environment variable. Using this and --cd it is possible to export from one CVS repository and import to another. =item --rev-root *experimental* Sets the root of the source tree to export. All files to be exported must be under this root directory. The default rev-root is all of the leading non-wildcard directory names. This can be useful in the unusual case of exporting a sub-tree of a larger project. I think. =item -r -r v_0_001:v_0_002 -f -r v_0_002:-f Passed to 'cvs log' as a '-r' revision specification. WARNING: if this string doesn't contain a ':', you're probably doing something wrong, since you're not specifying a revision range. vcp may warn about this in the future. This will normally only replicate files which are tagged. This means that files that have been added since, or which are missing the tag for some reason, are ignored. Use the L</-f> option to force files that don't contain the tag to be exported. This is probably what is expected. One of -r or L</-D> must be specified. =item -d -d "2000-11-18 5:26:30<=" Passed to 'cvs log' as a '-d' date specification. WARNING: if this string doesn't contain a '>' or '<', you're probably doing something wrong, since you're not specifying a range. vcp may warn about this in the future. One of L</-r> or -D must be specified. =item -f Not implemented yet. This option causes vcp to attempt to export files that don't contain a particular tag. It is an error to specify C<-f> without C<-r>. It does this by exporting all revisions of files between the oldest and newest files that the -r specified. Without C<-f>, these would be ignored. =back =head1 LIMITATIONS "What we have here is a failure to communicate!" - The warden in Cool Hand Luke CVS does not try to protect itself from people checking in things that look like snippets of CVS log file: they go in Ok, and they come out Ok, screwing up the parser. So, if you come accross a repository that contains messages that look like "cvs log" output, this is likely to go awry. At least one cvs repository out there has multiple revisions of a single file with the same rev number. The second and later revisions with the same rev number are ignored with a warning like "Can't add same revision twice:...". =cut $VERSION = 1.1 ; use strict ; use Carp ; use Getopt::Long ; use VCP::Debug ':debug' ; use Regexp::Shellish qw( :all ) ; use VCP::Rev ; use VCP::Source ; use base 'VCP::Source' ; use fields ( 'CVS_CUR', ## The current change number being processed 'CVS_FILESPEC', ## What revs of what files to get. ARRAY ref. 'CVS_BOOTSTRAP', ## Forces bootstrap mode 'CVS_IS_INCREMENTAL', ## Hash of filenames, 0->bootstrap, 1->incremental 'CVS_INFO', ## Results of the 'cvs --version' command and CVSROOT 'CVS_LABEL_CACHE', ## ->{$name}->{$rev} is a list of labels for that rev 'CVS_LABELS', ## Array of labels from 'p4 labels' 'CVS_MAX', ## The last change number needed 'CVS_MIN', ## The first change number needed 'CVS_REV_SPEC', ## The revision spec to pass to `cvs log` 'CVS_DATE_SPEC', ## The date spec to pass to `cvs log` 'CVS_FORCE_MISSING', ## Set if -f was specified. 'CVS_CVSROOT', ## What to pass with -d, if anything 'CVS_LOG_CARRYOVER', ## The unparsed bit of the log file 'CVS_LOG_FILE_DATA', ## Data about all revs of a file from the log file 'CVS_LOG_STATE', ## Parser state machine state 'CVS_LOG_REV', ## The revision being parsed (a hash) 'CVS_NAME_REP_NAME', ## A mapping of repository names to names, used to ## figure out what files to ignore when a cvs log ## goes ahead and logs a file which doesn't match ## the revisions we asked for. 'CVS_NEEDS_BASE_REV', ## What base revisions are needed. Base revs are ## needed for incremental (ie non-bootstrap) updates, ## which is decided on a per-file basis by looking ## at VCP::Source::is_bootstrap_mode( $file ) and ## the file's rev number (ie does it end in .1). 'CVS_SAW_EQUALS', ## Set when we see the ==== lines in log file [1] ) ; sub new { my $class = shift ; $class = ref $class || $class ; my VCP::Source::cvs $self = $class->SUPER::new( @_ ) ; ## Parse the options my ( $spec, $options ) = @_ ; my $parsed_spec = $self->parse_repo_spec( $spec ) ; my $files = $parsed_spec->{FILES} ; my $work_dir ; my $rev_root ; my $rev_spec ; my $date_spec ; my $force_missing ; ##TODO: Add option to Regexp::Shellish to allow '...' instead of or in ## addition to '**'. GetOptions( "b|bootstrap:s" => sub { my ( $name, $val ) = @_ ; $self->{CVS_BOOTSTRAP} = $val eq "" ? [ compile_shellish( "**" ) ] : [ map compile_shellish( $_ ), split /,+/, $val ] ; }, "cd=s" => \$work_dir, "rev-root=s" => \$rev_root, "r=s" => \$rev_spec, "d=s" => \$date_spec, "f+" => \$force_missing, ) or $self->usage_and_exit ; unless ( defined $rev_spec || defined $date_spec ) { print STDERR "Revision (-r) or date (-D) specification missing\n" ; $self->usage_and_exit ; } if ( $force_missing && ! defined $rev_spec ) { print STDERR "Force missing (-f) may not be used without a revision spec (-r)\n" ; $self->usage_and_exit ; } unless ( defined $rev_root ) { $self->deduce_rev_root( $files ) ; } else { $files = "$rev_root/$files" ; } ## TODO: Figure out whether we should make rev_root merely set the rev_root ## in the header. I think we probably should do it that way, as it's more ## flexible and less confusing. my $recurse = $files =~ s{/\.\.\.$}{} ; ## Don't normalize the filespec. $self->filespec( $files ) ; $self->cvsroot( $parsed_spec->{SERVER} ) ; $self->rev_spec( $rev_spec ) ; $self->date_spec( $date_spec ) ; $self->force_missing( $force_missing ) ; ## Make sure the cvs command is available $self->command( 'cvs', '-Q', '-z9' ) ; $self->command_stderr_filter( qr{^(?:cvs (?:server|add|remove): use 'cvs commit' to.*)\n} ) ; ## Doing a CVS command or two here also forces cvs to be found in new(), ## or an exception will be thrown. $self->command_dir( $work_dir ) if defined $work_dir ; $self->cvs( ['--version' ], \$self->{CVS_INFO} ) ; return $self ; } sub is_incremental { my VCP::Source::cvs $self= shift ; my ( $file, $first_rev ) = @_ ; my $bootstrap_mode = substr( $first_rev, -2 ) eq ".1" || ( $self->{CVS_BOOTSTRAP} && grep $file =~ $_, @{$self->{CVS_BOOTSTRAP}} ) ; return $bootstrap_mode ? 0 : "incremental" ; } sub filespec { my VCP::Source::cvs $self = shift ; $self->{CVS_FILESPEC} = shift if @_ ; return $self->{CVS_FILESPEC} ; } sub rev_spec { my VCP::Source::cvs $self = shift ; $self->{CVS_REV_SPEC} = shift if @_ ; return $self->{CVS_REV_SPEC} ; } sub rev_spec_cvs_option { my VCP::Source::cvs $self = shift ; return defined $self->rev_spec? "-r" . $self->rev_spec : (), } sub date_spec { my VCP::Source::cvs $self = shift ; $self->{CVS_DATE_SPEC} = shift if @_ ; return $self->{CVS_DATE_SPEC} ; } sub date_spec_cvs_option { my VCP::Source::cvs $self = shift ; return defined $self->date_spec ? "-d" . $self->date_spec : (), } sub force_missing { my VCP::Source::cvs $self = shift ; $self->{CVS_FORCE_MISSING} = shift if @_ ; return $self->{CVS_FORCE_MISSING} ; } sub cvsroot { my VCP::Source::cvs $self = shift ; $self->{CVS_CVSROOT} = shift if @_ ; return $self->{CVS_CVSROOT} ; } sub cvsroot_cvs_option { my VCP::Source::cvs $self = shift ; return defined $self->cvsroot ? "-d" . $self->cvsroot : (), } sub denormalize_name { my VCP::Source::cvs $self = shift ; return '/' . $self->SUPER::denormalize_name( @_ ) ; } sub handle_header { my VCP::Source::cvs $self = shift ; my ( $header ) = @_ ; $header->{rep_type} = 'cvs' ; $header->{rep_desc} = $self->{CVS_INFO} ; $header->{rev_root} = $self->rev_root ; $self->dest->handle_header( $header ) ; return ; } sub cvs { my VCP::Source::cvs $self = shift ; unshift @{$_[0]}, $self->cvsroot_cvs_option ; return $self->SUPER::cvs( @_ ) ; } sub get_rev { my VCP::Source::cvs $self = shift ; my VCP::Rev $r ; ( $r ) = @_ ; my $wp = $self->work_path( $r->name, $r->rev_id ) ; $r->work_path( $wp ) ; $self->mkpdir( $wp ) ; $self->cvs( [ "checkout", "-r" . $r->rev_id, "-p", $r->source_name, ], '>', $wp, ) ; } sub copy_revs { my VCP::Source::cvs $self = shift ; $self->{CVS_LOG_STATE} = '' ; $self->{CVS_LOG_CARRYOVER} = '' ; $self->revs( VCP::Revs->new ) ; ## We need to watch STDERR for messages like ## cvs log: warning: no revision `ch_3' in `/home/barries/src/revengine/tmp/cvsroot/foo/add/f4,v' ## and figure out which files should not be reported, because cvs log will ## emit the entire log for these files. my $tmp_f = $self->command_stderr_filter ; my %ignore_files ; my $ignore_file = sub { exists $ignore_files{$self->{CVS_NAME_REP_NAME}->{$_[0]}} ; } ; $self->command_stderr_filter( sub { my ( $err_text_ref ) = @_ ; $$err_text_ref =~ s@ ^cvs\slog:\swarning:\sno\srevision\s.*?\sin\s[`"'](.*)[`"']\n @ $ignore_files{$1} = undef ; '' ; @gxme ; } ) ; $self->{CVS_LOG_FILE_DATA} = {} ; $self->{CVS_LOG_REV} = {} ; $self->{CVS_SAW_EQUALS} = 0 ; $self->cvs( [ "log", $self->rev_spec_cvs_option, $self->date_spec_cvs_option, length $self->filespec ? $self->filespec : (), ], '>', sub { $self->parse_log_file( @_ ) }, ) ; $self->command_stderr_filter( $tmp_f ) ; my $revs = $self->revs ; ## Figure out the time stamp range for -f (FORCE_MISSING) calcs. my ( $min_rev_spec_time, $max_rev_spec_time ) ; if ( $self->{CVS_FORCE_MISSING} ) { ## If the rev_spec is /:$/ || /^:/, we tweak the range ends. my $max_time = 0 ; $max_rev_spec_time = 0 ; $min_rev_spec_time = 0 if substr( $self->rev_spec, 0, 1 ) eq ':' ; for my $r ( @{$revs->as_array_ref} ) { next if $r->is_base_rev ; my $t = $r->time ; $max_time = $t if $t >= $max_rev_spec_time ; next if $ignore_file->( $r->source_name ) ; $min_rev_spec_time = $t if $t <= ( $min_rev_spec_time || $t ) ; $max_rev_spec_time = $t if $t >= $max_rev_spec_time ; } $max_rev_spec_time = $max_time if substr( $self->rev_spec, -1 ) eq ':' ; debug( "cvs: -f including files in ['" . localtime( $min_rev_spec_time ), "'..'" . localtime( $max_rev_spec_time ), "']" ) if debugging $self ; } ## Remove extra revs from queue. ## TODO: Debug simultaneous use of -r and -D, since we probably are ## blowing away revs that -D included that -r didn't. I haven't ## checked to see if we do or don't blow said revs away. my %oldest_revs ; $self->revs( VCP::Revs->new ) ; for my $r ( @{$revs->as_array_ref} ) { if ( $ignore_file->( $r->source_name ) ) { if ( defined $min_rev_spec_time && $r->time >= $min_rev_spec_time && $r->time <= $max_rev_spec_time ) { debug( "cvs: -f including file '", $r->source_name, "'" ) if debugging $self ; } else { debug( "cvs: ignoring file '", $r->source_name, "': no revisions match -r" ) if debugging $self ; next ; } } ## Because of the order of the log file, the last rev set is always ## the first rev in the range. $oldest_revs{$r->source_name} = $r ; $self->revs->add( $r ) ; } $revs = $self->revs ; ## Add in base revs for my $fn ( keys %oldest_revs ) { my $r = $oldest_revs{$fn} ; my $rev_id = $r->rev_id ; if ( $self->is_incremental( $fn, $rev_id ) ) { $rev_id =~ s{(\d+)$}{$1-1}e ; $revs->add( VCP::Rev->new( source_name => $r->source_name, name => $r->name, rev_id => $rev_id, type => $r->type, ) ) } } $self->dest->sort_revs( $self->revs ) ; my VCP::Rev $r ; while ( $r = $self->revs->shift ) { $self->get_rev( $r ) ; $self->dest->handle_rev( $r ) ; } } # Here's a typical file log entry. # ############################################################################### # #RCS file: /var/cvs/cvsroot/src/Eesh/Changes,v #Working file: src/Eesh/Changes #head: 1.3 #branch: #locks: strict #access list: #symbolic names: # Eesh_003_000: 1.3 # Eesh_002_000: 1.2 # Eesh_000_002: 1.1 #keyword substitution: kv #total revisions: 3; selected revisions: 3 #description: #---------------------------- #revision 1.3 #date: 2000/04/22 05:35:27; author: barries; state: Exp; lines: +5 -0 #*** empty log message *** #---------------------------- #revision 1.2 #date: 2000/04/21 17:32:14; author: barries; state: Exp; lines: +22 -0 #Moved a bunch of code from eesh, then deleted most of it. #---------------------------- #revision 1.1 #date: 2000/03/24 14:54:10; author: barries; state: Exp; #*** empty log message *** #============================================================================= ############################################################################### sub parse_log_file { my ( $self, $input ) = @_ ; if ( defined $input ) { $self->{CVS_LOG_CARRYOVER} .= $input ; } else { ## There can only be leftovers if they don't end in a "\n". I've never ## seen that happen, but given large comments, I could be surprised... $self->{CVS_LOG_CARRYOVER} .= "\n" if length $self->{CVS_LOG_CARRYOVER} ; } my $store_rev = sub { # my ( $is_oldest ) = @_ ; return unless keys %{$self->{CVS_LOG_REV}} ; $self->{CVS_LOG_REV}->{MESSAGE} = '' if $self->{CVS_LOG_REV}->{MESSAGE} eq '*** empty log message ***' ; $self->{CVS_LOG_REV}->{MESSAGE} =~ s/\r\n|\n\r/\n/g ; #debug map "$_ => $self->{CVS_LOG_FILE_DATA}->{$_},", sort keys %{$self->{CVS_LOG_FILE_DATA}} ; $self->_add_rev( $self->{CVS_LOG_FILE_DATA}, $self->{CVS_LOG_REV} ) ; # if ( $is_oldest ) { # if ( # $self->is_incremental( # $self->{CVS_LOG_FILE_DATA}->{WORKING}, # $self->{CVS_LOG_REV}->{REV} # ) # ) { # $self->{CVS_LOG_REV}->{REV} =~ s{(\d+)$}{$1-1}e ; # # $self->_add_rev( # $self->{CVS_LOG_FILE_DATA}, # $self->{CVS_LOG_REV}, # "is base rev" # ); # } # } $self->{CVS_LOG_REV} = {} ; } ; local $_ ; ## DOS, Unix, Mac lineends spoken here. while ( $self->{CVS_LOG_CARRYOVER} =~ s/^(.*(?:\r\n|\n\r|\n))// ) { $_ = $1 ; ## [1] See bottom of file for a footnote explaining this delaying of ## clearing CVS_LOG_FILE_DATA and CVS_LOG_STATE until we see ## a ========= line followed by something other than a ----------- ## line. ## TODO: Move to a state machine design, hoping that all versions ## of CVS emit similar enough output to not trip it up. ## TODO: BUG: Turns out that some CVS-philes like to put text ## snippets in their revision messages that mimic the equals lines ## and dash lines that CVS uses for delimiters!! PLEASE_TRY_AGAIN: if ( /^===========================================================*$/ ) { $store_rev->() ;# "is oldest" ) ; $self->{CVS_SAW_EQUALS} = 1 ; next ; } if ( /^----------------------------*$/ ) { $store_rev->() unless $self->{CVS_SAW_EQUALS} ; $self->{CVS_SAW_EQUALS} = 0 ; $self->{CVS_LOG_STATE} = 'rev' ; next ; } if ( $self->{CVS_SAW_EQUALS} ) { $self->{CVS_LOG_FILE_DATA} = {} ; $self->{CVS_LOG_STATE} = '' ; $self->{CVS_SAW_EQUALS} = 0 ; } unless ( $self->{CVS_LOG_STATE} ) { if ( /^(RCS file|Working file|head|branch|locks|access list|keyword substitution):\s*(.*)/i ) { #warn uc( (split /\s+/, $1 )[0] ), "/", $1, ": ", $2, "\n" ; $self->{CVS_LOG_FILE_DATA}->{uc( (split /\s+/, $1 )[0] )} = $2 ; #$DB::single = 1 if /keyword/ && $self->{CVS_LOG_FILE_DATA}->{WORKING} =~ /Makefile/ ; } elsif ( /^total revisions:\s*([^;]*)/i ) { $self->{CVS_LOG_FILE_DATA}->{TOTAL} = $1 ; if ( /selected revisions:\s*(.*)/i ) { $self->{CVS_LOG_FILE_DATA}->{SELECTED} = $1 ; } } elsif ( /^symbolic names:/i ) { $self->{CVS_LOG_STATE} = 'tags' ; next ; } elsif ( /^description:/i ) { $self->{CVS_LOG_STATE} = 'desc' ; next ; } else { carp "Unhandled CVS log line '$_'" if /\S/ ; } } elsif ( $self->{CVS_LOG_STATE} eq 'tags' ) { if ( /^\S/ ) { $self->{CVS_LOG_STATE} = '' ; goto PLEASE_TRY_AGAIN ; } my ( $tag, $rev ) = m{(\S+):\s+(\S+)} ; unless ( defined $tag ) { carp "Can't parse tag from CVS log line '$_'" ; $self->{CVS_LOG_STATE} = '' ; next ; } $self->{CVS_LOG_FILE_DATA}->{TAGS}->{$tag} = $rev ; push( @{$self->{CVS_LOG_FILE_DATA}->{RTAGS}->{$rev}}, $tag ) ; } elsif ( $self->{CVS_LOG_STATE} eq 'rev' ) { ( $self->{CVS_LOG_REV}->{REV} ) = m/([\d.]+)/ ; $self->{CVS_LOG_STATE} = 'rev_meta' ; next ; } elsif ( $self->{CVS_LOG_STATE} eq 'rev_meta' ) { for ( split /;\s*/ ) { my ( $key, $value ) = m/(\S+):\s+(.*?)\s*$/ ; $self->{CVS_LOG_REV}->{uc($key)} = $value ; } $self->{CVS_LOG_STATE} = 'rev_message' ; next ; } elsif ( $self->{CVS_LOG_STATE} eq 'rev_message' ) { $self->{CVS_LOG_REV}->{MESSAGE} .= $_ ; } } ## Never, ever forget the last rev. "Wait for me! Wait for me!" ## Most of the time, this should not be a problem: cvs log puts a ## line of "=" at the end. But just in case I don't know of a ## funcky condition where that might not happen... unless ( defined $input ) { $store_rev->() ; # "is oldest" ) ; $self->{CVS_LOG_REV} = undef ; $self->{CVS_LOG_FILE_DATA} = undef ; } } # Here's a (probably out-of-date by the time you read this) dump of the args # for _add_rev: # ############################################################################### #$file = { # 'WORKING' => 'src/Eesh/eg/synopsis', # 'SELECTED' => '2', # 'LOCKS' => 'strict', # 'TOTAL' => '2', # 'ACCESS' => '', # 'RCS' => '/var/cvs/cvsroot/src/Eesh/eg/synopsis,v', # 'KEYWORD' => 'kv', # 'RTAGS' => { # '1.1' => [ # 'Eesh_003_000', # 'Eesh_002_000' # ] # }, # 'HEAD' => '1.2', # 'TAGS' => { # 'Eesh_002_000' => '1.1', # 'Eesh_003_000' => '1.1' # }, # 'BRANCH' => '' #}; #$rev = { # 'DATE' => '2000/04/21 17:32:16', # 'MESSAGE' => 'Moved a bunch of code from eesh, then deleted most of it. #', # 'STATE' => 'Exp', # 'AUTHOR' => 'barries', # 'REV' => '1.1' #}; ############################################################################### sub _add_rev { my VCP::Source::cvs $self = shift ; my ( $file_data, $rev_data, $is_base_rev ) = @_ ; my $norm_name = $self->normalize_name( $file_data->{WORKING} ) ; my $action = $rev_data->{STATE} eq "dead" ? "delete" : "edit" ; my $type = $file_data->{KEYWORD} =~ /[o|b]/ ? "binary" : "text" ; #debug map "$_ => $rev_data->{$_}, ", sort keys %{$rev_data} ; my VCP::Rev $r = VCP::Rev->new( source_name => $file_data->{WORKING}, name => $norm_name, rev_id => $rev_data->{REV}, type => $type, # ! $is_base_rev # ? ( action => $action, time => $self->parse_time( $rev_data->{DATE} ), user_id => $rev_data->{AUTHOR}, comment => $rev_data->{MESSAGE}, state => $rev_data->{STATE}, labels => $file_data->{RTAGS}->{$rev_data->{REV}}, # ) # : (), ) ; $self->{CVS_NAME_REP_NAME}->{$file_data->{WORKING}} = $file_data->{RCS} ; eval { $self->revs->add( $r ) ; } ; if ( $@ ) { if ( $@ =~ /Can't add same revision twice/ ) { warn $@ ; } else { die $@ ; } } } =head1 SUBCLASSING This class uses the fields pragma, so you'll need to use base and possibly fields in any subclasses. =head1 COPYRIGHT Copyright 2000, Perforce Software, Inc. All Rights Reserved. This module and the VCP package are licensed according to the terms given in the file LICENSE accompanying this distribution, a copy of which is included in L<vcp>. =head1 AUTHOR Barrie Slaymaker <barries@slaysys.com> =cut ## FOOTNOTES: # [1] :pserver:guest@cvs.tigris.org:/cvs hass some goofiness like: #---------------------------- #revision 1.12 #date: 2000/09/05 22:37:42; author: thom; state: Exp; lines: +8 -4 # #merge revision history for cvspatches/root/log_accum.in #---------------------------- #revision 1.11 #date: 2000/08/30 01:29:38; author: kfogel; state: Exp; lines: +8 -4 #(derive_subject_from_changes_file): use \t to represent tab #characters, not the incorrect \i. #============================================================================= #---------------------------- #revision 1.11 #date: 2000/09/05 22:37:32; author: thom; state: Exp; lines: +3 -3 # #merge revision history for cvspatches/root/log_accum.in #---------------------------- #revision 1.10 #date: 2000/07/29 01:44:06; author: kfogel; state: Exp; lines: +3 -3 #Change all "Tigris" ==> "Helm" and "tigris" ==> helm", as per Daniel #Rall's email about how the tigris path is probably obsolete. #============================================================================= #---------------------------- #revision 1.10 #date: 2000/09/05 22:37:23; author: thom; state: Exp; lines: +22 -19 # #merge revision history for cvspatches/root/log_accum.in #---------------------------- #revision 1.9 #date: 2000/07/29 01:12:26; author: kfogel; state: Exp; lines: +22 -19 #tweak derive_subject_from_changes_file() #============================================================================= #---------------------------- #revision 1.9 #date: 2000/09/05 22:37:13; author: thom; state: Exp; lines: +33 -3 # #merge revision history for cvspatches/root/log_accum.in # 1
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#131 | 5403 | barrie_slaymaker | - Misc logging, maintainability & debugging improvements | ||
#130 | 4517 | barrie_slaymaker |
- VCP::Source::cvs uses earlier_ids to prevent deletes from occuring before clones or branches |
||
#129 | 4507 | barrie_slaymaker |
- RevML: - added <action>, removed <delete>, <placeholder> and <move> - added <from_id> for clones (and eventually merge actions) - Simplified DTD (can't branch DTD based on which action any more) - VCP::Source::cvs, VCP::Filter::changesets and VCP::Dest::p4 support from_id in <action>clone</action> records - VCP::Dest::perl_data added - VCP::Rev::action() "branch" added, no more undefined action strings - "placeholder" action removed |
||
#128 | 4487 | barrie_slaymaker | - dead code removal (thanks to clkao's coverage report) | ||
#127 | 4476 | barrie_slaymaker | - misc bugfixes | ||
#126 | 4220 | barrie_slaymaker | - VCP::Source::cvs no longer passes a leading '/' on module name for checkout | ||
#125 | 4039 | barrie_slaymaker |
- VCP::Source::scan_metadata() API now in place, - VCP::Source::copy_revs() is fully deprecated. |
||
#124 | 4035 | barrie_slaymaker |
- VCP::Source::cvs no longer specifies a user_id or time for concocted delete revs |
||
#123 | 4034 | barrie_slaymaker | - VCP::Source::cvs does not set the user_id on branch revs | ||
#122 | 4032 | barrie_slaymaker | - VCP::Dest::p4 now estimates missing metadata | ||
#121 | 4021 | barrie_slaymaker |
- Remove all phashes and all base & fields pragmas - Work around SWASHGET error |
||
#120 | 4012 | barrie_slaymaker | - Remove dependance on pseudohashes (deprecated Perl feature) | ||
#119 | 4005 | barrie_slaymaker | - VCP::Source::cvs: minor, abandoned code removed | ||
#118 | 3999 | barrie_slaymaker | - VCP::Source::cvs parses branch numbers more correcly | ||
#117 | 3995 | barrie_slaymaker |
- VCP::Source::cvs parses large RCS text sections with lots of "@@" escapes. |
||
#116 | 3994 | barrie_slaymaker |
- VCP::Source::cvs splits dead revs in to edit followed by a delete rev if need be (not always, but only when needed). |
||
#115 | 3993 | barrie_slaymaker | - Fold in changes from clkao's SVN work | ||
#114 | 3982 | barrie_slaymaker |
- VCP::Source no longer leaks memory by delete()ing from a phash - VCP::Source::cvs now flushes to disk more often to conserve RAM |
||
#113 | 3979 | barrie_slaymaker |
- VCP::Source::cvs branch number regex fixed - VCP::Dest::null --dont-get-revs option added |
||
#112 | 3970 | barrie_slaymaker |
- VCP::Source handles rev queing, uses disk to reduce RAM - Lots of other fixes |
||
#111 | 3930 | barrie_slaymaker |
- VCP::Source::cvs and VCP::Dest::p4 handle cloning deletes - "placeholder" actions and is_placeholder_rev() deprecated in favor of is_branch_rev() and is_clone_rev(). - Misc cleanups and minor bugfixes |
||
#110 | 3923 | barrie_slaymaker |
- VCP::Source::cvs now uses source_...() where possible to avoid using modified fields (for instance fields touched by Map:) |
||
#109 | 3916 | barrie_slaymaker | - Reduce memory consumption | ||
#108 | 3907 | barrie_slaymaker | - Debugging cleanups | ||
#107 | 3904 | barrie_slaymaker |
- VCP::Source::cvs copes with consecutive dead revisions by concocting edit revisions with the same rev_id but different ids. This is experimental. |
||
#106 | 3894 | barrie_slaymaker | - VCP::Source::cvs RCS scan RE no longer explodes | ||
#105 | 3855 | barrie_slaymaker |
- vcp scan, filter, transfer basically functional - Need more work in re: storage format, etc, but functional |
||
#104 | 3850 | barrie_slaymaker | - No longer stores all revs in memory | ||
#103 | 3836 | barrie_slaymaker | - Sources no longer cache all revs in RAM before sending | ||
#102 | 3819 | barrie_slaymaker | - Factor send & queueing of revs up in to VCP::Source | ||
#101 | 3818 | barrie_slaymaker | - VCP::Source::{cvs,p4,vsS} use less memory | ||
#100 | 3813 | barrie_slaymaker | - VCP::Rev::previous() is no more | ||
#99 | 3811 | barrie_slaymaker | - fetch_*() and get_rev() renamed get_source_file() | ||
#98 | 3804 | barrie_slaymaker | - Refactored to prepare way for reducing memory footprint | ||
#97 | 3800 | barrie_slaymaker | - <branches> removed from all code | ||
#96 | 3763 | barrie_slaymaker |
- VCP::Source::cvs now explains what was ignored and what was scanned when it finds both foo,v and Attic/foo,v. |
||
#95 | 3747 | barrie_slaymaker | - VCP::Source::cvs branches vendor tags properly | ||
#94 | 3746 | barrie_slaymaker |
- VCP::Source::cvs parses vendor tags when no revisions are present on the vendor branch (as per Marc Tooley's patch) - add test for said parsing |
||
#93 | 3744 | barrie_slaymaker |
- VCP::Source::cvs understands a source filespec of "..." for local respositories (unless --use-cvs) |
||
#92 | 3681 | barrie_slaymaker | - VCP now scans much more of real_vss_1 and converts it to revml | ||
#91 | 3677 | barrie_slaymaker |
- rev_root sanity check is now case insensitive on Win32 - Parens in source filespecs are now treated as regular characters, not capture groups - ** is not treated as '...' |
||
#90 | 3568 | barrie_slaymaker | - Use xchdir() instead of chdir() | ||
#89 | 3532 | john_fetkovich |
changed File::Spec->rel2abs( blah, start_dir ) to start_dir_rel2abs blah everywhere. which does the same thing and is defined in VCP::Utils |
||
#88 | 3523 | john_fetkovich | more ui defaults and checks added | ||
#87 | 3489 | barrie_slaymaker | - Document options emitted to .vcp files. | ||
#86 | 3477 | barrie_slaymaker | - Make --rev-root only available in VCP::Source::p4 | ||
#85 | 3462 | barrie_slaymaker | - Make sure bootstrap regexps get compiled | ||
#84 | 3460 | barrie_slaymaker |
- Revamp Plugin/Source/Dest hierarchy to allow for reguritating options in to .vcp files |
||
#83 | 3418 | barrie_slaymaker |
- Better progress reporting. - VCP::Source::cvs now actually passes the -k option through to cvs |
||
#82 | 3384 | john_fetkovich | moved setting of default repo_id | ||
#81 | 3285 | john_fetkovich |
In 'sub new' constructor, Only call parse_cvs_repo_spec if a $spec is provided. parse_cvs_repo_spec also now sets repo_id. |
||
#80 | 3274 | john_fetkovich | split part of 'sub new' into 'sub init' | ||
#79 | 3206 | john_fetkovich | documentation changes | ||
#78 | 3205 | john_fetkovich | pod improvements | ||
#77 | 3199 | john_fetkovich | Improved documentation of --bootstrap switch. | ||
#76 | 3167 | barrie_slaymaker |
Add profiling report that details various chunks of time taken. |
||
#75 | 3166 | barrie_slaymaker | Remove stale code, update to _run3 calling conventions | ||
#74 | 3155 | barrie_slaymaker |
Convert to logging using VCP::Logger to reduce stdout/err spew. Simplify & speed up debugging quite a bit. Provide more verbose information in logs. Print to STDERR progress reports to keep users from wondering what's going on. Breaks test; halfway through upgrading run3() to an inline function for speed and for VCP specific features. |
||
#73 | 3133 | barrie_slaymaker |
Make destinations call back to sources to check out files to simplify the architecture (is_metadata_only() no longer needed) and make it more optimizable (checkouts can be batched). |
||
#72 | 3131 | barrie_slaymaker |
Double the speed of the RCS file parser. Deprecate VCP::Revs::shift() in favor of remove_all(). |
||
#71 | 3129 | barrie_slaymaker |
Stop calling the slow Cwd::cwd so much, use start_dir instead. |
||
#70 | 3120 | barrie_slaymaker | Move changeset aggregation in to its own filter. | ||
#69 | 3106 | barrie_slaymaker |
Remove an unused field (state) from VCP::Rev optimize and bugfix labelmap |
||
#68 | 3096 | barrie_slaymaker | Tuning | ||
#67 | 3086 | barrie_slaymaker |
Optimize change aggregation from something like O(N^2) down to something more reasonable. Noticable only on large transfers. |
||
#66 | 3081 | barrie_slaymaker |
Get cvs->p4 propogation branches with multiple tags working to spec. |
||
#65 | 3075 | barrie_slaymaker |
Make all empty branches be timestamped at $last_rev_time_in_cvsroot + 1 second. |
||
#64 | 3068 | barrie_slaymaker | Note a cleanup to be done someday | ||
#63 | 3067 | barrie_slaymaker | Improve revision linking logic for direct-read case | ||
#62 | 3061 | barrie_slaymaker | Make VCP use the first label for a branch instead of the last | ||
#61 | 3038 | barrie_slaymaker | Get proper identification of founding revisions implemented. | ||
#60 | 3035 | barrie_slaymaker | code format tweak. | ||
#59 | 3032 | barrie_slaymaker | Fix rev_id parsing RE | ||
#58 | 3013 | barrie_slaymaker | Clean up minor undefined var warning discovered in testing | ||
#57 | 3010 | barrie_slaymaker |
Log the number of tag applications (the xzfree86 repo has a lot of tags applied to each file rev, I need numbers). |
||
#56 | 3007 | barrie_slaymaker | Read CVS vendor branche tags | ||
#55 | 2982 | barrie_slaymaker | Treat 1.0, 2.0, 3.0 as first revs | ||
#54 | 2979 | barrie_slaymaker |
Put all 1.x, 2.x, 3.x, etc. on the main dev trunk (1.1.1.x, 1.1.2.x, etc. are still separate branches) |
||
#53 | 2973 | barrie_slaymaker | Fix handling of branched but unchanged files | ||
#52 | 2972 | barrie_slaymaker | Interim checkin | ||
#51 | 2938 | john_fetkovich | added empty() calls | ||
#50 | 2925 | barrie_slaymaker | Source cleanup; no significant changes | ||
#49 | 2900 | barrie_slaymaker | Handle case where the first rev in a branch is deleted. | ||
#48 | 2824 | john_fetkovich |
removed CVS_CONTINUE field from Source/cvs.pm, and added CONTINUE field and continue accessor to Source.pm. Moved parsing of the --continue option also. |
||
#47 | 2809 | barrie_slaymaker |
Implement --repo-id in Plugin.pm, refactor source & dest options parsing starting in VCP::Source::cvs (need to roll out to other sources and dests), get t/91cvs2revml.t passing again (first time in months! branching and --continue support works in cvs->foo!). |
||
#46 | 2802 | john_fetkovich |
Added a source_repo_id to each revision, and repo_id to each Source and Dest. The repo_ids include repository type (cvs,p4,revml,vss,...) and the repo_server fields. Changed the $self->...->set() and $self->...->get() lines in VCP::Dest::* to pass in a conglomerated key value, by passing in the key as an ARRAY ref. Also various restructuring in VCP::DB.pm, VCP::DB_file.pm and VCP::DB_file::sdbm.pm related to this change. |
||
#45 | 2800 | barrie_slaymaker | Get --continue working in cvs->foo transfers. | ||
#44 | 2743 | john_fetkovich |
Add fields to vcp: source_name, source_filebranch_id, source_branch_id, source_rev_id, source_change_id 1. Alter revml.dtd to include the fields 2. Alter bin/gentrevml to emit legal RevML 3. Extend VCP::Rev to have the fields 4. Extend VCP::{Source,Dest}::revml to read/write the fields (VCP::Dest::revml should die() if VCP tries to emit illegal RevML) 5. Extend VCP::{Source,Dest}::{cvs,p4} to read the fields 7. Get all tests through t/91*.t to pass except those that rely on ch_4 labels |
||
#43 | 2667 | barrie_slaymaker | Convert more to IPC::Run3 | ||
#42 | 2389 | john_fetkovich |
removed calls to methods: command_stderr_filter command_ok_result_codes command_chdir and replaced with named Plugin::run_safely method parameters stderr_filter ok_result_codes in_dir respectively, where possible. |
||
#41 | 2337 | barrie_slaymaker | Correct the parser, reduce memory usage | ||
#40 | 2331 | barrie_slaymaker | tune memory usage of VCP::Source::cvs | ||
#39 | 2322 | barrie_slaymaker | Fix jack-in-the-bug options parsing exposed by .vcp files | ||
#38 | 2321 | barrie_slaymaker | Fix a jack-in-the-bug triggered by changing gentrevml's time outputs. | ||
#37 | 2293 | barrie_slaymaker | Update CHANGES, TODO, improve .vcp files, add --init-cvs | ||
#36 | 2267 | barrie_slaymaker | factor out cvs2revml, test both --use-cvs and direct modes, with times | ||
#35 | 2266 | barrie_slaymaker | clean up --use-cvs doc | ||
#34 | 2245 | barrie_slaymaker | cvs -r (re)implemented for direct reads, passes all cvs-only tests | ||
#33 | 2241 | barrie_slaymaker | RCS file scanning improvements, implement some of -r | ||
#32 | 2240 | barrie_slaymaker | Start on cvs -r option support. | ||
#31 | 2236 | barrie_slaymaker | Debug, speed up cvs file parsing | ||
#30 | 2235 | barrie_slaymaker | Debugging cvs speed reader. | ||
#29 | 2228 | barrie_slaymaker | working checkin | ||
#28 | 2199 | barrie_slaymaker | um, comment out the cache I was using to debug. | ||
#27 | 2153 | barrie_slaymaker | checkin | ||
#26 | 2151 | barrie_slaymaker | checkin | ||
#25 | 2042 | barrie_slaymaker | Basic source::p4 branching support | ||
#24 | 2026 | barrie_slaymaker | VCP::8::cvs now supoprt branching | ||
#23 | 2009 | barrie_slaymaker |
lots of fixes, improve core support for branches and VCP::Source::cvs now supports branches. |
||
#22 | 2006 | barrie_slaymaker |
more preparations for branching support, handling of cvs :foo:... CVSROOT specs, misc fixes, improvements |
||
#21 | 1998 | barrie_slaymaker | Initial, revml and core VCP support for branches | ||
#20 | 1855 | barrie_slaymaker | Major VSS checkin. Works on Win32 | ||
#19 | 1728 | barrie_slaymaker | CVS on win32, minor bugfixes | ||
#18 | 1367 | barrie_slaymaker | lots of docco updates | ||
#17 | 1358 | barrie_slaymaker | Win32 changes | ||
#16 | 1330 | barrie_slaymaker | Ignore cvs lock mgmt warnings in VCP::Source::cvs. | ||
#15 | 723 | barrie_slaymaker | VCP::Dest::cvs tuning and cvs and p4 bugfixes | ||
#14 | 705 | barrie_slaymaker | Release 0.22. | ||
#13 | 692 | barrie_slaymaker |
Add VCP::Utils::p4 and use it to get VCP::Dest::p4 to create it's own client view as needed. |
||
#12 | 689 | barrie_slaymaker |
reinstate -f behavior as the default for VCP::Source::cvs, clean up -D --> -d doco. |
||
#11 | 687 | barrie_slaymaker | remove -f, tweak deduce_rev_root | ||
#10 | 630 | barrie_slaymaker |
Fix bug in CVS log file parsing that made it think it was always seeing the same file, different revisions over and over again. Reported by Matthew Attaway. |
||
#9 | 628 | barrie_slaymaker | Cleaned up POD in bin/vcp, added BSD-style license. | ||
#8 | 627 | barrie_slaymaker | Beef up CVS log file parsing. | ||
#7 | 626 | barrie_slaymaker | Removed POD that was older than the current feature set. | ||
#6 | 625 | barrie_slaymaker | Add NOTE about required " " in cvs options. | ||
#5 | 624 | barrie_slaymaker | Add a space to bin/vcp SYNOPSIS after the cvs -r option. | ||
#4 | 480 | barrie_slaymaker |
0.06 Wed Dec 20 23:19:15 EST 2000 - bin/vcp: Added --versions, which loads all modules and checks them for a $VERSION and print the results out. This should help with diagnosing out-of-sync modules. - Added $VERSION vars to a few modules :-). Forgot to increment any $VERSION strings. - VCP::Dest::cvs: The directory "deeply" was not being `cvs add`ed on paths like "a/deeply/nested/file", assuming "deeply" had no files in it. - VCP::Dest::revml: fixed a bug that was causing files with a lot of linefeeds to be emitted in base64 instead of deltaed. This means most text files. - Various minor cleanups of diagnostics and error messages, including exposing "Can't locate Foo.pm" when a VCP::Source or VCP::Dest module depends on a module that's not installed, as reported by Jeff Anton. |
||
#3 | 478 | barrie_slaymaker |
0.05 Mon Dec 18 07:27:53 EST 2000 - Use `p4 labels //...@label` command as per Rober Cowham's suggestion, with the '-s' flag recommended by Christopher Siewald and Amaury.FORGEOTDARC@atsm.fr. Though it's actually something like vcp: running /usr/bin/p4 -u safari -c safari -p localhost:5666 -s files //.../NtLkly //...@compiler_a3 //.../NtLkly //...@compiler_may3 and so //on //for 50 parameters to get the speed up. I use the //.../NtLkly "file" as //a separator between the lists of files in various //revisions. Hope nobody has any files named that :-). What I should do is choose a random label that doesn't occur in the labels list, I guess. - VCP::Source::revml and VCP::Dest::revml are now binary, control code, and "hibit ASCII" (I know, that's an oxymoron) clean. The <comment>, <delta>, and <content> elements now escape anything other than tab, line feed, space, or printable chars (32 <= c <= ASCII 126) using a tag like '<char code="0x09">'. The test suite tests all this. Filenames should also be escaped this way, but I didn't get to that. - The decision whether to do deltas or encode the content in base64 is now based on how many characters would need to be escaped. - We now depend on the users' diff program to have a "-a" option to force it to diff even if the files look binary to it. I need to use Diff.pm and adapt it for use on binary data. - VCP::Dest::cvs now makes sure that no two consecutive revisions of the same file have the same mod_time. VCP::Source::p4 got so fast at pulling revisions from the repositories the test suite sets up that CVS was not noticing that files had changed. - VCP::Plugin now allows you to set a list of acceptable result codes, since we now use p4 in ways that make it return non-zero result codes. - VCP::Revs now croaks if you try to add two entries of the same VCP::Rev (ie matching filename and rev_id). - The <type> tag is now limited to "text" or "binary", and is meant to pass that level of info between foreign repositories. - The <p4_info> on each file now carries the one line p4 description of the file so that p4->p4 transferes can pick out the more detailed info. VCP::Source::p4, VCP::Dest::p4 do this. - VCP::{Source,Dest}::{p4,cvs} now set binaryness on added files properly, I think. For p4->p4, the native p4 type is preserved. For CVS sources, seeing the keyword substitution flag 'o' or 'b' implies binaryness, for p4, seeing a filetype like qr/u?x?binary/ or qr/x?tempobj/ or "resource" implies binaryness (to non-p4 destinations). NOTE: Seeing a 'o' or 'b' in a CVS source only ends up setting the 'b' option on the destination. That should be ok for most uses, but we can make it smarter for cvs->cvs transfers if need be. |
||
#2 | 468 | barrie_slaymaker |
- VCP::Dest::p4 now does change number aggregation based on the comment field changing or whenever a new revision of a file with unsubmitted changes shows up on the input stream. Since revisions of files are normally sorted in time order, this should work in a number of cases. I'm sure we'll need to generalize it, perhaps with a time thresholding function. - t/90cvs.t now tests cvs->p4 replication. - VCP::Dest::p4 now doesn't try to `p4 submit` when no changes are pending. - VCP::Rev now prevents the same label from being applied twice to a revision. This was occuring because the "r_1"-style label that gets added to a target revision by VCP::Dest::p4 could duplicate a label "r_1" that happened to already be on a revision. - Added t/00rev.t, the beginnings of a test suite for VCP::Rev. - Tweaked bin/gentrevml to comment revisions with their change number instead of using a unique comment for every revision for non-p4 t/test-*-in-0.revml files. This was necessary to test cvs->p4 functionality. |
||
#1 | 467 | barrie_slaymaker | Version 0.01, initial checkin in perforce public depot. |