package VCP::Dest::p4 ; =head1 NAME VCP::Dest::p4 - p4 destination driver =head1 SYNOPSIS vcp <source> p4:user:password@p4port:<dest> vcp <source> p4:user(client):password@p4port:<dest> vcp <source> p4:<dest> =head1 DESCRIPTION The <dest> spec is a perforce repository spec and must begin with // and a depot name ("//depot"), not a local filesystem spec or a client spec. There should be a trailing "/..." specified. If no user name, password, or port are given, the underlying p4 command will look at that standard environment variables. VCP sets the environment P4PASSWD rather than giving p4 the password on the command line so it won't be logged in debugging or error messages. The other options are passed on the command line. If no client name is given, a temporary client name like "vcp_tmp_1234" will be created and used. The P4CLIENT environment variable will not be used. If an existing client name is given, the named client spec will be saved off, altered, used, and restored. If the client was created for this import, it will be deleted when complete, regardless of whether the client was specified by the user or was randomly generated. WARNING: If perl coredumps or is killed with a signal that prevents cleanup--like a SIGKILL (9)--the the client deletion or restoral will not occur. The client view is not saved on disk, either, so back it up manually if you care. THE CLIENT SAVE/RESTORE FEATURE IS EXPERIMENTAL AND MAY CHANGE BASED ON USER FEEDBACK. VCP::Dest::p4 attempts change set aggregation by sorting incoming revisions. See L<VCP::Dest/rev_cmp_sub> for the order in which revisions are sorted. Once sorted, a change is submitted whenever the change number (if present) changes, the comment (if present) changes, or a new rev of a file with the same name as a revision that's pending. THIS IS EXPERIMENTAL, PLEASE DOUBLE CHECK EVERYTHING! =head1 OPTIONS =for test_scripts t/90revml2p4_0.t t/90revml2p4_1.t t/90revml2p4_2.t t/90revml2p4_3.t =over =item --run-p4d Runs a p4d instance in the directory indicated by repo_server (use a directory path rather than a host name). If repo_server contains a port, that port will be used, otherwise a random port will be used. Dies unless the directory exists and contains files matching db.* (to help prevent unexpected initializing of empty directories). VCP will kill this p4d when it's done. =item --init-p4d Initializes a directory and starts a p4d in it on the given port. Refuses to init a non-empty directory. In this case the p4port portion of the destination specification must point to a directory; and the port, if present, will be used for the port (otherwise a randomized port number other than p4d's 1666 default will be used.) A temporary p4d will be started that should be shut down upon process exit. If the process does not exit cleanly (for instance, if sent the QUIT signal), this shutdown may not occur. =item --delete-p4d-dir If C<--init-p4d> is passed and the target directory is not empty, it will be removed before running the p4d. THIS IS DANGEROUS AND SHOULD ONLY BE USED IN TEST ENVIRONMENTS. =item --change-branch-rev-1 Some SCMs don't create a branch of a file until there is actually a change made to that file. So the first revision of a file on a branch is different from its parent on the main branch. Normally, p4 does not work this way: rev #1 of a branched file is a very inexpensive copy of the parent revision: you do a p4 integrate, submit, edit, submit sequence to branch a file and introduce it's changes. This option forces VCP to do a p4 integrate, add, submit sequence to branch files, thus capturing the branch and the file alterations in one change. Using this option allows VCP to more exactly model the source repository in the destination repository revision-for-revision, but leaves you with a perforce repository that may not be consistent with your work practices, so it is not the default behavior. =back =head1 NOTES The p4 destination driver allows branching from deleted revisions of files to accomodate source repositories that allow it. It does this by branching from the revision prior to the deleted revision. The p4 destination driver also allows the first revision of a file to be a deleted revision by forcing an empty "add" followed by a "delete". CVS does this on the main trunk (rev 1.1 is "dead", ie deleted) when you add a file on a branch. =cut $VERSION = 1 ; use strict ; use vars qw( $debug ) ; $debug = 0 ; use Carp ; use File::Basename ; use File::Path ; use VCP::Debug ':debug' ; use VCP::Dest ; use VCP::Logger qw( lg lg_fh pr ); use VCP::Rev ; use VCP::Utils qw( empty ); use VCP::Utils::p4 qw( underscorify_name ); ## If we ever want to store state in the dest repo, this constant ## turns that on. It should become an option if it is ever ## reenabled, probably replacing the VCP::RevMapDB. use constant store_state_in_repo => 0; use base qw( VCP::Dest VCP::Utils::p4 ) ; use fields ( # 'P4_SPEC', ## The root of the tree to update 'P4_PENDING', ## Revs pending the next submit 'P4_WORK_DIR', ## Where to do the work. 'P4_REPO_CLIENT', ## See VCP::Utils::p4 for accessors and usage... 'P4_INIT_P4D', ## --init-p4d flag 'P4_DELETE_P4D_DIR', ## --delete_p4d flag (delete temp p4d dir before starting p4d) 'P4_RUN_P4D', ## --run_p4d flag (run temp p4d) 'P4_LABEL_FORM', ## A cached label form 'P4_ADDED_LABELS', ## A hash of labels that we've already added. ## TODO: Preload this using the p4 labels command ## to save some time when writing to big repos? 'P4_CHANGE_BRANCH_REV_1', ## Which command sequence to use when ## branching a file. See POD above. ## members for change number divining: 'P4_PREV_CHANGE_ID', ## The change_id in the r sequence, if any 'P4_PREV_COMMENT', ## Used to detect change boundaries 'P4_REV_MAP', ## RevMapDB ) ; sub new { my $class = shift ; $class = ref $class || $class ; my VCP::Dest::p4 $self = $class->SUPER::new( @_ ) ; ## Parse the options my ( $spec, $options ) = @_ ; $self->parse_p4_repo_spec( $spec ) unless empty $spec; $self->parse_options( $options, "init-p4d!" => \$self->{P4_INIT_P4D}, "change-branch-rev-1!" => \$self->{P4_CHANGE_BRANCH_REV_1}, "delete-p4d-dir!" => \$self->{P4_DELETE_P4D_DIR}, "run-p4d!" => \$self->{P4_RUN_P4D}, ); return $self ; } sub init { my VCP::Dest::p4 $self = shift ; my $repo_server = $self->repo_server; $repo_server = $ENV{P4PORT} unless defined $repo_server; die 'P4PORT not set\n' if empty $repo_server; $self->repo_id( "p4:$repo_server" ) if empty $self->repo_id; ## We use the rev_root only to munge branch specs. ## We let p4 set the rev_root by setting the ## client view to the destination path the user specified, so as perforce ## adds each file in our working dir, it puts them in the right spot ## in the repository (under the destination rev_root). $self->deduce_rev_root( $self->repo_filespec ); if ( $self->{P4_INIT_P4D} ) { if ( $self->{P4_DELETE_P4D_DIR} ) { $self->rev_map->delete_db; $self->head_revs->delete_db; $self->files->delete_db; } $self->init_p4d; } else { pr "ignoring --delete-p4d-dir, which is only useful with --init-p4d\n" if $self->{P4_DELETE_P4D_DIR}; $self->run_p4d if $self->{P4_RUN_P4D}; } $self->rev_map->open_db; $self->head_revs->open_db; $self->files->open_db; $self->set_up_p4_user_and_client; $self->init_p4_view ; } sub init_p4d { my $self = shift; my ( $dir, $port ) = split ":", $self->repo_server, 2; my @files; @files = glob "$dir/*" if -d $dir; if ( @files && $self->{P4_DELETE_P4D_DIR} ) { require File::Path; rmtree [ @files ]; @files = glob "$dir/*"; } die "cannot --init-p4d on non-empty dir $dir\n" if @files; $self->mkdir( $dir ) unless -e $dir; ## VCPP4LICENSE env var, if present, points to a p4 license file to ## use. Link to it. my $license = $ENV{VCPP4LICENSE} ; if ( ! empty $license && -r $license && ! -f "$dir/license" ) { symlink $license, "$dir/license" or die "failed to link '$dir/license' to '$license' (p4 license)"; pr "linked '$dir/license' to '$license' (p4 license)\n"; } ## Ok, this is wierd: we need to fork & run p4d in foreground mode ## so that we can capture it's PID and kill it later. There ## doesn't seem to be the equivalent of a 'p4d.pid' file. If we let ## it daemonize, then I don't know how to get it's PID. $port = $self->launch_p4d( $dir, $port ); $self->repo_server( "localhost:$port" ); } sub checkout_file { my VCP::Dest::p4 $self = shift ; my VCP::Rev $r ; ( $r ) = @_ ; confess unless defined $self && defined $self->header ; debug "retrieving '", $r->as_string, "' from p4 dest repo" if debugging; ## The rev_root was put in the client view, p4 will "denormalize" ## the name for us. my $work_path = $self->work_path( $r->name ) ; debug "work_path '$work_path'" if debugging; my ( undef, $work_dir ) = fileparse( $work_path ) ; $self->mkpdir( $work_path ) unless -d $work_dir ; my $tag = store_state_in_repo ? "\@vcp_" . underscorify_name $r->id : "#" . ($self->rev_map->get( [ $r->source_repo_id, $r->id ] ))[1]; ## The -f forces p4 to sync even if it thinks it doesn't have to. It's ## not in there for any known reason, just being conservative. $self->p4( ['sync', '-f', $r->name . $tag ] ) ; die "'$work_path' not created in backfill" unless -e $work_path ; return $work_path ; } sub handle_header { my VCP::Dest::p4 $self = shift ; my ( $h ) = @_; $self->{P4_PENDING} = [] ; $self->{P4_PREV_COMMENT} = undef ; $self->{P4_PREV_CHANGE_ID} = undef ; $self->{P4_LABEL_FORM} = undef ; $self->{P4_ADDED_LABELS} = {}; $self->SUPER::handle_header( @_ ) ; if ( $h->{branches} ) { for my $b ( $h->{branches}->get ) { my $spec = $b->p4_branch_spec; next if empty $spec; ## Re-root the view. my $found_it; $spec = $self->build_p4_form( map { if ( $found_it ) { ( my $source_root = $h->{ rev_root} ) =~ s{^/*}{//}; ( my $dest_root = $self->rev_root ) =~ s{^/*}{//}; s{\Q$source_root}{$dest_root}g; undef $found_it; } $found_it = $_ eq "View"; $_; } $self->parse_p4_form( $spec ) ); $self->p4( [qw(branch -i -f)], \$spec ); } } } sub handle_rev { my VCP::Dest::p4 $self = shift ; my VCP::Rev $r ; ( $r ) = @_ ; debug "handle_rev got ", $r->as_string if debugging; my $change_id = $r->change_id; if ( @{$self->{P4_PENDING}} && ! $self->{P4_CHANGE_BRANCH_REV_1} ? $change_id ne $self->{P4_PREV_CHANGE_ID} && ( debugging() ? debug "time to submit: change_id changed" : 1 ) : ( defined $r->comment && defined $self->{P4_PREV_COMMENT} && $r->comment ne $self->{P4_PREV_COMMENT} && ( debugging() ? debug "time to submit: comment changed [", $r->comment, "] vs [", $self->{P4_PREV_COMMENT}, "]" : 1 ) ) || ( ! $r->is_placeholder_rev && grep( ! $_->is_placeholder_rev && $r->name eq $_->name, @{$self->{P4_PENDING}} ) && ( debugging() ? debug "time to submit: name repeated" : 1 ) ) || ( $r->previous && grep( ! $_->is_placeholder_rev && $_ == $r->previous, @{$self->{P4_PENDING}} ) && ( debugging() ? debug "time to submit: need a previous revision for ", "\n", " ", $r->as_string, "\n", map " " . $_->as_string . "\n", grep( $_ == $r->previous, @{$self->{P4_PENDING}} ) : 1 ) ) ) { $self->submit; $self->{P4_PREV_COMMENT} = $r->comment; } $self->{P4_PREV_CHANGE_ID} = $change_id; ## TODO: batch these too so that we can pass them all to ## fetch_files() and allow the source to perhaps operate ## more efficiently. if ( $r->is_base_rev ) { my ( $work_path ) = VCP::Revs->fetch_files( $r ); $self->compare_base_revs( $r, $work_path ); return; } push @{$self->{P4_PENDING}}, $r; } sub _add_it { my VCP::Dest::p4 $self = shift; my VCP::Rev $r ; ( $r ) = @_ ; my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; ! -e $work_path; } sub handle_footer { my VCP::Dest::p4 $self = shift ; $self->submit if @{$self->{P4_PENDING}}; $self->SUPER::handle_footer ; } { my $lg_fh = lg_fh; my $change_spec; sub p4_submit { my VCP::Dest::p4 $self = shift; my ( $max_time, $description, @revs ) = @_; unless ( $change_spec ) { $self->p4( [ "change", "-o" ], undef, \$change_spec ) ; ## ASSume the Files & Description are the last two fields in ## the spec. $change_spec =~ s/^(Description|Files):.*\r?\n\r?.*//ms or die "couldn't remove change file list and description\n$change_spec" ; } if ( length $description ) { $description =~ s/^/\t/gm ; chomp $description; } my $files = join "", map "\t//" . $self->denormalize_name( $_->name ) . "\n", @revs; my $change = $change_spec . <<END_SPEC; Description: $description Files: $files END_SPEC my $submit_log; $self->p4([ "submit", "-i"], \$change, \$submit_log ) ; ## extract the change number and the file list from the submit output print $lg_fh $submit_log; my $change_number; my %p4_rev_ids; { while ( $submit_log =~ m{\G(.*)([\r\n]+|\z)}g ) { my $line = $1; if ( $line =~ m{^\w+\s+//(.*)#(\d+)\z} ) { $p4_rev_ids{$1} = $2; } elsif ( $line =~ m{^Change (\d+) } ) { $change_number = $1; } } } pr "p4 change $change_number submitted" if defined $change_number; ## Force the correct date and a user id { my $repl = "Change:\t$change_number\n\n"; $repl .= "Date:\t$max_time\n\n" if defined $max_time; $change =~ s/^Change:.*\r?\n/$repl/m or die "couldn't modify change number:\n$change"; $change =~ s/^Status:.*\r?\n/Status: submitted\n\n/m or die "couldn't modify status:\n$change"; ## ASSume the files list is last $change =~ s/^Files:.*\r?\n//ms or die "couldn't remove files list:\n$change"; my $user_id; for ( @{$self->{P4_PENDING}} ) { unless ( empty $_->user_id ) { $user_id = underscorify_name( $_->user_id ); last; } } if ( defined $user_id ) { $change =~ s/^User:.*/User:\t$user_id/m or die "couldn't modify change user\n$change" ; } $self->p4( [ "change", "-i", "-f" ], \$change ) ; } for my $r ( @revs ) { my $denorm_name = $self->denormalize_name( $r->name ); my $rev_id = $p4_rev_ids{$denorm_name}; unless ( defined $rev_id ) { $submit_log =~ s/^/ /mg; require Data::Dumper; die "no rev number found in p4 submit output for ", $r->as_string, ":\n", $submit_log, "looked for: $denorm_name\n", "p4 revs parsed: ", Data::Dumper::Dumper( \%p4_rev_ids ); } $self->rev_map->set( [ $r->source_repo_id, $r->id ], $denorm_name, $rev_id, $self->files->get( [ $denorm_name ] ), ); $self->head_revs->set( [ $r->source_repo_id, $r->source_filebranch_id ], $r->source_rev_id ); } } } my $branch_spec; sub _integrate_pending_revs { my VCP::Dest::p4 $self = shift ; my @to_integrate; my @to_resolve; my $max_time; for my $r ( @{$self->{P4_PENDING}} ) { my $pr = $r->previous; if ( $pr ## && $self->_add_it( $r ) && ( $pr->branch_id || "" ) ne ( $r->branch_id || "" ) ) { ## Branch the previous version to make the new one. Leave ## add_it set so we can drop the new one in over the ## branched version in case it's changed. my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; my $denorm_name = $self->denormalize_name( $fn ); my ( $pfull_name, $prev_id, $pstate ) = $self->rev_map->get( [ $pr->source_repo_id, $pr->id ] ); die "Can't integrate from unknown revision:\n", "parent: ", $pr->as_string, "\n", "child: ", $r->as_string, "\n" unless defined $pstate; if ( $pstate eq "deleted" ) { ## CVS allows branching from "deleted" revisions, ## let's hope that the revision before the previous ## revision is *not* deleted. $prev_id--; } push @to_integrate, [ $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id, ]; $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; } } return unless @to_integrate; unless ( $branch_spec ) { $self->p4( [ "branch", "-o", "vcp_$$" ], \undef, \$branch_spec ); $branch_spec =~ s/^(View:).*/$1\n/ms; } my $this_branch_spec = join "", $branch_spec, map { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$_; qq{\t "//$pfull_name" "//$denorm_name"\n}; } @to_integrate; $self->p4( [ "branch", "-i" ], \$this_branch_spec ); $self->p4_x( [ "integrate", "-b", "vcp_$$" ], [ map { my ( $denorm_name, $prev_id ) = @{$_}[2,5]; "//$denorm_name#$prev_id\n" } @to_integrate ] ); my @revs_to_submit; my @to_submit_now; for my $i ( @to_integrate ) { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$i; push @to_resolve, $fn if $self->files->exists( [ $denorm_name ] ); $self->files->set( [ $denorm_name ], "integrated to" ); if ( $self->{P4_CHANGE_BRANCH_REV_1} ) { ## Force it to be `p4 add`ed as well by _add_edit_delete() if ( ! $r->is_placeholder_rev || grep( ($_->previous||0) == $r, @{$self->{P4_PENDING}} ) ) { ## There's a rev that the _add_edit_delete logic will ## see and do the add, so delete this rev. debug "unlinking $work_path so it will be added\n" if debugging; unlink $work_path or die "$! unlinking $work_path\n" if -e $work_path; } else { ## There's no rev in this change that the _add_edit_delete ## logic will see, so make sure this gets added in the main ## p4 submit invocation. Don't do it right now because we ## don't want to create an additional change number. push @revs_to_submit, $r; } } else { ## Integrate the first change push @to_submit_now, $i; } } $self->p4( [ "resolve", "-at" ] ) if @to_resolve; if ( @to_submit_now ) { $self->p4_submit( $max_time, "create branches", map $_->[0], @to_submit_now ); for my $i ( @to_submit_now ) { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$i; lg "syncing recently integrated file $fn"; $self->p4( [ "sync", "$fn" ] ) unless -e $work_path; } } return @revs_to_submit; } sub _pre_add_pending_initial_deletes { my VCP::Dest::p4 $self = shift ; my @to_add; my $max_time; for my VCP::Rev $r ( @{$self->{P4_PENDING}} ) { next if $r->is_placeholder_rev; if ( $r->action eq "delete" ) { my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; my $denorm_name = $self->denormalize_name( $fn ); my ( @state ) = $self->files->get( [ $denorm_name ] ); unless ( @state ) { ## Never seen this file before. ## ## This could be a fake deletion from CVS, which does this when ## a file is added on a branch. So we fake up an empty ## file and add it. lg "creating '$work_path' so that it may be deleted"; $self->mkpdir( $work_path ); open F, ">$work_path" or die "$! opening $work_path\n"; close F; $r->dest_work_path( $work_path ) ; push @to_add, [ $r, $denorm_name ]; $self->files->set( [ $denorm_name ], "touched and added" ); $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; } } } if ( @to_add ) { $self->p4_x( [ "add" ], [ map $_->[0]->name . "\n", @to_add ] ); $self->p4_submit( $max_time, "spoof addition of deleted file", map $_->[0], @to_add ); } } ## Revs are only queued up by handle_rev() because we need to ## deal with integrate commands before we deal with other commands. sub _add_edit_delete_pending { my VCP::Dest::p4 $self = shift; my @to_delete; my %to_add; ## Keyed on filetype my %to_edit; ## Keyed on filetype my @to_link; my @to_submit; for my VCP::Rev $r ( @{$self->{P4_PENDING}} ) { next if $r->is_placeholder_rev; push @to_submit, $r; my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; debug "work_path '$work_path'" if debugging; my $denorm_name = $self->denormalize_name( $r->name ); if ( $r->action eq "delete" ) { my ( @state ) = $self->files->get( [ $denorm_name ] ); my $deleteit = $state[0] ne "deleted"; if ( $deleteit && ! -e $work_path ) { $self->p4( [ "sync", "-f", $fn ] ); } if ( -e $work_path ) { unlink $work_path or die "$! unlinking $work_path" ; } push @to_delete, $fn if $deleteit; $self->files->set( [ $denorm_name ], "deleted" ); } else { ## add or edit my $filetype = defined $r->p4_info && $r->p4_info =~ /\((\S+)\)$/ ? $1 : $r->type ; if ( $self->_add_it( $r ) ) { $self->mkpdir( $work_path ) ; push @{$to_add{$filetype}}, $r; $self->files->set( [ $denorm_name ], "added" ); } else { push @{$to_edit{$filetype}}, $r; $self->files->set( [ $denorm_name ], "edited" ); } my $work_path = $self->work_path( $r->name ) ; $r->dest_work_path( $work_path ) ; push @to_link, $r; ## TODO: Provide command line options for user-defined tag prefixes $r->add_label( "vcp_" . $r->id ) if store_state_in_repo; } } $self->p4_x( [ "delete" ], [ map "$_\n", @to_delete ] ) if @to_delete; for ( keys %to_edit ) { my @to_edit = @{$to_edit{$_}}; $self->p4_x( [ "edit", "-t", $_ ], [ map $_->name . "\n", @to_edit ] ); for my $r ( @to_edit ) { my $work_path = $self->work_path( $r->name ) ; unlink $self->work_path( $r->name ) or die "$! unlinking $work_path" } } my @source_fns = VCP::Revs->fetch_files( @to_link ); for my $r ( @to_link ) { my $source_fn = shift @source_fns; my $work_path = $r->dest_work_path; if ( $source_fn ne $work_path ) { debug "linking $source_fn to $work_path" if debugging; link $source_fn, $work_path or die "$! linking '$source_fn' -> '$work_path'" ; } if ( defined $r->mod_time ) { utime $r->mod_time, $r->mod_time, $work_path or die "$! changing times on $work_path" ; } } $self->p4_x( [ "add", "-t", $_ ], [ map $_->name . "\n", @{$to_add{$_}} ] ) for keys %to_add; return @to_submit; } sub submit { my VCP::Dest::p4 $self = shift ; ## Take care of any integrations up front so that they're ## present if the first rev on a branch has changes on it so it ## can be "p4 edit"ed in the main submit. my @to_submit = $self->_integrate_pending_revs; ## CVS can create initial revs that are deleted; p4 must add them ## then delete them. $self->_pre_add_pending_initial_deletes( $_ ); push @to_submit, $self->_add_edit_delete_pending; if ( @to_submit ) { my %pending_labels ; my %comments ; my $max_time ; if ( @{$self->{P4_PENDING}} ) { for my $r ( @{$self->{P4_PENDING}} ) { $comments{$r->comment} = $r->name if defined $r->comment ; $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; for my $l ( $r->labels ) { push @{$pending_labels{$l}}, $r->dest_work_path ; } } if ( defined $max_time ) { my @f = reverse( (localtime $max_time)[0..5] ) ; $f[0] += 1900 ; ++$f[1] ; ## Day of month needs to be 1..12 $max_time = sprintf "%04d/%02d/%02d %02d:%02d:%02d", @f ; } elsif ( debugging ) { debug "No max_time found" ; } } my $description = join( "\n", keys %comments ) ; $self->p4_submit( $max_time, $description, @to_submit ); ## Create or add a label spec for each of the labels. The 'sort' is to ## make debugging output more legible. ## TODO: Modify RevML to allow label metadata (owner, desc, options) ## to be passed through. Same for user, client, jobs metadata etc. ## The assumption is made that most labels will apply to a single change ## number, so we do the labelling once per submit. I don't think that ## this will break if it doesn't, but TODO: add more labelling tests. pr "storing " . keys( %pending_labels ) . " labels" if keys %pending_labels > 25; for my $label ( sort keys %pending_labels ) { my $p4_label = underscorify_name $label; $self->{P4_ADDED_LABELS}->{$p4_label} ||= do { $self->{P4_LABEL_FORM} ||= do { my $out; $self->p4( [qw( label -o ), $p4_label], undef, \$out ) ; $out =~ s/(^.+:\s+)\Q$p4_label\E$/$1<<<LABEL>>>/gm; $out; }; ( my $l_desc = $self->{P4_LABEL_FORM} ) =~ s/<<<LABEL>>>/$p4_label/g; $self->p4( [qw( label -i ) ], \$l_desc, \my $dev_null ) ; 1; }; $self->p4_x( [qw( labelsync -a -l ), $p4_label ], [ map "$_\n", @{$pending_labels{$label}} ], \my $dev_null ) ; } } ## Make GC and file cleanup occur ASAP for ( @{$self->{P4_PENDING}} ) { $_->previous( undef ); $_->set_work_path( undef ) if defined $_->work_path; } @{$self->{P4_PENDING}} = () ; } ## Prevent VCP::Plugin from rmtree-ing the workspace we're borrowing sub DESTROY { my VCP::Dest::p4 $self = shift ; $self->work_root( undef ) ; $self->SUPER::DESTROY ; } =head1 AUTHOR Barrie Slaymaker <barries@slaysys.com> =head1 COPYRIGHT Copyright (c) 2000, 2001, 2002 Perforce Software, Inc. All rights reserved. See L<VCP::License|VCP::License> (C<vcp help license>) for the terms of use. =cut 1
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#129 | 5402 | barrie_slaymaker | - Disallow "?" in p4 names | ||
#128 | 4521 | barrie_slaymaker | - Debugging lg() calls removed | ||
#127 | 4520 | barrie_slaymaker |
- VCP::Dest::p4 another "No file(s) to resolve" error squashed - This occured when cloning or branching a file over a previously deleted file |
||
#126 | 4507 | barrie_slaymaker |
- RevML: - added <action>, removed <delete>, <placeholder> and <move> - added <from_id> for clones (and eventually merge actions) - Simplified DTD (can't branch DTD based on which action any more) - VCP::Source::cvs, VCP::Filter::changesets and VCP::Dest::p4 support from_id in <action>clone</action> records - VCP::Dest::perl_data added - VCP::Rev::action() "branch" added, no more undefined action strings - "placeholder" action removed |
||
#125 | 4489 | barrie_slaymaker | - removed commented-out debug code | ||
#124 | 4476 | barrie_slaymaker | - misc bugfixes | ||
#123 | 4416 | barrie_slaymaker | - VCP::Dest::p4 tries not to send pre-epoch timestamps to p4 | ||
#122 | 4411 | barrie_slaymaker | - missing metadata is logged in the log file | ||
#121 | 4166 | barrie_slaymaker | - No more VCP::Dest::p4 BUG on labelling a freshly branch file | ||
#120 | 4165 | barrie_slaymaker |
- VCP::Source::null now returns /dev/null or NUL for files - VCP::Dest::p4 now watches is_branch_rev() to detect branches - t/99p4_label_branch_rev_1.t reproduces a bug reported by Matt Attaway < matt at perforce com > |
||
#119 | 4033 | barrie_slaymaker | - VCP::Dest::p4 now really estimates time | ||
#118 | 4032 | barrie_slaymaker | - VCP::Dest::p4 now estimates missing metadata | ||
#117 | 4021 | barrie_slaymaker |
- Remove all phashes and all base & fields pragmas - Work around SWASHGET error |
||
#116 | 4012 | barrie_slaymaker | - Remove dependance on pseudohashes (deprecated Perl feature) | ||
#115 | 4006 | barrie_slaymaker | - VCP will now use the P4::Client module if it's installed. | ||
#114 | 4003 | barrie_slaymaker |
- VCP::Dest::p4 now prevents illegal filenames, user_ids, and label names |
||
#113 | 3997 | barrie_slaymaker | - Limit p4 integrate size to 100 files | ||
#112 | 3975 | barrie_slaymaker |
- VCP::Dest::p4 can now branch the same file multiple times in one change. |
||
#111 | 3970 | barrie_slaymaker |
- VCP::Source handles rev queing, uses disk to reduce RAM - Lots of other fixes |
||
#110 | 3930 | barrie_slaymaker |
- VCP::Source::cvs and VCP::Dest::p4 handle cloning deletes - "placeholder" actions and is_placeholder_rev() deprecated in favor of is_branch_rev() and is_clone_rev(). - Misc cleanups and minor bugfixes |
||
#109 | 3916 | barrie_slaymaker | - Reduce memory consumption | ||
#108 | 3907 | barrie_slaymaker | - Debugging cleanups | ||
#107 | 3903 | barrie_slaymaker | - Minor debug logging impovements | ||
#106 | 3893 | barrie_slaymaker | - p4d is now launched in the proper directory | ||
#105 | 3855 | barrie_slaymaker |
- vcp scan, filter, transfer basically functional - Need more work in re: storage format, etc, but functional |
||
#104 | 3850 | barrie_slaymaker | - No longer stores all revs in memory | ||
#103 | 3837 | barrie_slaymaker | - Improved progress bar support | ||
#102 | 3813 | barrie_slaymaker | - VCP::Rev::previous() is no more | ||
#101 | 3812 | barrie_slaymaker | - VCP::Dest::* no longer need VCP::Rev->previous() | ||
#100 | 3811 | barrie_slaymaker | - fetch_*() and get_rev() renamed get_source_file() | ||
#99 | 3805 | barrie_slaymaker | - VCP::Revs::fetch_files() removed | ||
#98 | 3754 | barrie_slaymaker |
- VCP::Dest::p4 no longer tries to label files the current change just deleted. |
||
#97 | 3749 | barrie_slaymaker | - error message tweaked | ||
#96 | 3748 | barrie_slaymaker | - VCP::Dest::p4 dies when trying to branch to/from same file | ||
#95 | 3741 | barrie_slaymaker | - Minor message cleanup | ||
#94 | 3712 | barrie_slaymaker | - Bogus confess() doesn't bite any more | ||
#93 | 3706 | barrie_slaymaker | - VCP gives some indication of output progress (need more) | ||
#92 | 3647 | barrie_slaymaker |
- All UI prompts & descriptions rewritten. - Minor tweak to VCP::Dest::p4 P4USER defaulting |
||
#91 | 3572 | john_fetkovich | added y/n question to accept default of user_id | ||
#90 | 3569 | barrie_slaymaker |
- Work around bug caused by p4's using the long pathname when $ENV{PWD} is not set. |
||
#89 | 3518 | john_fetkovich | more interactive ui improvements | ||
#88 | 3498 | barrie_slaymaker | - Fix changeset boundary detection bug in VCP::Dest::p4 | ||
#87 | 3491 | barrie_slaymaker | - All sections are now documented in generated config files | ||
#86 | 3460 | barrie_slaymaker |
- Revamp Plugin/Source/Dest hierarchy to allow for reguritating options in to .vcp files |
||
#85 | 3442 | barrie_slaymaker | - Win32 regex fixup for label specs | ||
#84 | 3432 | barrie_slaymaker | - "p4 print" to "p4 sync" | ||
#83 | 3423 | barrie_slaymaker | - Use the new VCP::Utils::p4::split_repo_server() | ||
#82 | 3411 | barrie_slaymaker |
- p4 output parsing improvment - clearer log messages |
||
#81 | 3402 | barrie_slaymaker |
- now passes all tests using the p4 api library. (still not default, set env var VCPP4API=1) - foo->p4 handles branch-but-no-change case when --change-branch-rev-1 is passed. - sources & dests can now provide their own command execution routine in place of shelling out to an external command (as in call the p4api library instead of running the p4 command). |
||
#80 | 3382 | john_fetkovich |
Moved setting of repo_id to 'sub init' moved defaulting of repo_server to P4PORT env var to 'sub init' |
||
#79 | 3284 | john_fetkovich |
'sub new' constructor in Source and Dest p4.pm fixed so parse_p4_repo_spec only called when a $spec is provided to the constructor. parse_p4_repo_spec now also sets the repo_id. parse_repo_spec (TODO item) no longer returns a hash value of the values parsed, it only sets fields in $self. Fixed a few places where that return hash was used. |
||
#78 | 3277 | john_fetkovich | split out 'sub init' from 'sub new' | ||
#77 | 3207 | john_fetkovich | doc (pod) fixes | ||
#76 | 3187 | barrie_slaymaker |
Tweak p4 change -o replacement logic to work with command line client. |
||
#75 | 3186 | barrie_slaymaker | cleanup | ||
#74 | 3185 | barrie_slaymaker | Reduce number of p4 changes commands to 1 per p4 submit. | ||
#73 | 3169 | barrie_slaymaker |
Log the stdout of all p4 submits use -x - for p4 integrate calls. Avoid Carp; use BUG. |
||
#72 | 3155 | barrie_slaymaker |
Convert to logging using VCP::Logger to reduce stdout/err spew. Simplify & speed up debugging quite a bit. Provide more verbose information in logs. Print to STDERR progress reports to keep users from wondering what's going on. Breaks test; halfway through upgrading run3() to an inline function for speed and for VCP specific features. |
||
#71 | 3139 | barrie_slaymaker | Batch calls to p4 integrate | ||
#70 | 3133 | barrie_slaymaker |
Make destinations call back to sources to check out files to simplify the architecture (is_metadata_only() no longer needed) and make it more optimizable (checkouts can be batched). |
||
#69 | 3127 | barrie_slaymaker | Minor cleanups. | ||
#68 | 3120 | barrie_slaymaker | Move changeset aggregation in to its own filter. | ||
#67 | 3117 | barrie_slaymaker |
Cut over to faster VCP::Rev::new, remove symbolic method calls. |
||
#66 | 3104 | john_fetkovich |
Added handling of VCPP4LICENSE environment variable in Dest::p4.pm. If that's present and pointing to a readable file when a p4 daemon is started, a symlink will be created in the (possibly newly created) p4root directory to point to the p4 license file pointed to by VCPP4LICENSE. The 'make' target test_all_p4_versions will cycle through each version of p4 and p4d contained in the 'p4versions' directory, in both unlicensed and (if VCPP4LICENSE present) licensed mode. |
||
#65 | 3098 | barrie_slaymaker |
Convert all length p4 command line calls to us p4 -x -. All hail p4 -x -. |
||
#64 | 3081 | barrie_slaymaker |
Get cvs->p4 propogation branches with multiple tags working to spec. |
||
#63 | 3076 | barrie_slaymaker | Improve change aggregation | ||
#62 | 3066 | barrie_slaymaker |
Set time correctly on integrate and pre-delete add changes. |
||
#61 | 3031 | barrie_slaymaker |
Fix empty branch creation batch adds, edits, and deletes rearrange for code maintainability |
||
#60 | 3029 | barrie_slaymaker |
Improve state database (does not affect operation, but was causing some interesting information to be missing when using bin/dump_* |
||
#59 | 3021 | barrie_slaymaker |
Force integrate to branch from proper revision and allow it to branch from the predecessor of a deleted file (CVS allows file deletes). |
||
#58 | 3011 | barrie_slaymaker | Free memory and delete files more aggressively | ||
#57 | 2981 | barrie_slaymaker | quash some output | ||
#56 | 2980 | barrie_slaymaker |
Don't let placeholders count as a needed previous revision when splitting the rev stream in to changes. |
||
#55 | 2976 | barrie_slaymaker |
Force submittal of parent revision if a child rev of it arrives before it is submitted. |
||
#54 | 2973 | barrie_slaymaker | Fix handling of branched but unchanged files | ||
#53 | 2972 | barrie_slaymaker | Interim checkin | ||
#52 | 2930 | john_fetkovich | added empty() calls | ||
#51 | 2926 | john_fetkovich |
remove --state-location switch add --db-dir and --repo-id switches build state location from concatenation of those two. |
||
#50 | 2915 | barrie_slaymaker |
Default to *not* changing the first rev of a branch, add --change-branch-rev-1 to enable the non-default behavior (which is what is used by most of the test suite). |
||
#49 | 2913 | barrie_slaymaker | Put a user id on each change | ||
#48 | 2912 | barrie_slaymaker | Remove spurious warn()ing | ||
#47 | 2910 | barrie_slaymaker | Update the submit time of each change. | ||
#46 | 2908 | barrie_slaymaker | Suppress "refreshed" messages | ||
#45 | 2907 | barrie_slaymaker |
Use persistent state file to track what's been seen so that delete/add/edit decisions work accross incremental imports |
||
#44 | 2838 | john_fetkovich | Use parse_options rather than using Getopt::Long directly. | ||
#43 | 2802 | john_fetkovich |
Added a source_repo_id to each revision, and repo_id to each Source and Dest. The repo_ids include repository type (cvs,p4,revml,vss,...) and the repo_server fields. Changed the $self->...->set() and $self->...->get() lines in VCP::Dest::* to pass in a conglomerated key value, by passing in the key as an ARRAY ref. Also various restructuring in VCP::DB.pm, VCP::DB_file.pm and VCP::DB_file::sdbm.pm related to this change. |
||
#42 | 2774 | barrie_slaymaker | Update HeadRevDB on submit/commit/write | ||
#41 | 2770 | barrie_slaymaker | tweak dump utils: sort output for readability | ||
#40 | 2721 | barrie_slaymaker | VCP::Dest::p4 now uses RevMapDB | ||
#39 | 2698 | barrie_slaymaker |
Don't add r_ and ch_ labels, switch to underscorify_name() to get standardized escaping of illegal p4 labels (still need to discover what, exactly, a legal p4 label name is). |
||
#38 | 2664 | barrie_slaymaker | Try IPC::Run3 to see if it speeds up p4 calls | ||
#37 | 2641 | barrie_slaymaker |
Add --run-p4d option to VCP::{Source,Dest}::p4. Implement port hunting and p4d up & ready detection for vcp-launched p4ds. |
||
#36 | 2589 | john_fetkovich |
Split 90p4.t into 90revml2p4_0.t, 90revml2p4_1.t, 91p42revml.t, 95p42cvs.t. Added some utilities to the library files listed. |
||
#35 | 2350 | barrie_slaymaker |
Get p4d to be killed after the last p4 cleanup commands, generalize the p4d launching & destruction system because we'll need it in VCP::Source::p4 too to take checkpoints. |
||
#34 | 2344 | barrie_slaymaker | Make the labelsync code more self-documenting | ||
#33 | 2336 | barrie_slaymaker | Notify user when shutting down p4d | ||
#32 | 2333 | barrie_slaymaker | Tweak logging of p4d command | ||
#31 | 2332 | barrie_slaymaker | Add --init-p4d and --delete-p4d-dir options | ||
#30 | 2322 | barrie_slaymaker | Fix jack-in-the-bug options parsing exposed by .vcp files | ||
#29 | 2276 | barrie_slaymaker |
optimize away extra p4 label -o / -i calls for speed, elegance update t/90p4.t |
||
#28 | 2059 | barrie_slaymaker | Support for branching in p4->p4 added | ||
#27 | 2049 | barrie_slaymaker | Get branching working in Dest::p4, clean up some tests. | ||
#26 | 2042 | barrie_slaymaker | Basic source::p4 branching support | ||
#25 | 2009 | barrie_slaymaker |
lots of fixes, improve core support for branches and VCP::Source::cvs now supports branches. |
||
#24 | 1855 | barrie_slaymaker | Major VSS checkin. Works on Win32 | ||
#23 | 1809 | barrie_slaymaker | VCP::Patch should ignore lineends | ||
#22 | 1367 | barrie_slaymaker | lots of docco updates | ||
#21 | 1055 | barrie_slaymaker |
add sorting, revamp test suite, misc cleanup. Dest/revml is not portable off my system yet (need to release ...::Diff) |
||
#20 | 827 | barrie_slaymaker | Add a test for and debug p4->cvs incremental exports. | ||
#19 | 825 | barrie_slaymaker |
test, handle case where no revs are transferred and VCP::Dest::*::handle_footer() blew up. |
||
#18 | 824 | barrie_slaymaker | Fix p4 backfilling path math to not mangle paths. | ||
#17 | 812 | barrie_slaymaker |
A more sensible name for on_first_rev(), and allow incremental foo->p4 updates to work by backfilling the base rev instead of trying to add it. |
||
#16 | 723 | barrie_slaymaker | VCP::Dest::cvs tuning and cvs and p4 bugfixes | ||
#15 | 705 | barrie_slaymaker | Release 0.22. | ||
#14 | 703 | barrie_slaymaker | VCP::Source::p4 now uses VCP::Utils::p4::parse_p4_repo_spec() | ||
#13 | 701 | barrie_slaymaker | Fixed VCP::Dest::p4 re-rooting problem, further t/* cleanup | ||
#12 | 700 | barrie_slaymaker | Delete orphan code | ||
#11 | 692 | barrie_slaymaker |
Add VCP::Utils::p4 and use it to get VCP::Dest::p4 to create it's own client view as needed. |
||
#10 | 689 | barrie_slaymaker |
reinstate -f behavior as the default for VCP::Source::cvs, clean up -D --> -d doco. |
||
#9 | 674 | barrie_slaymaker |
Fix paths passed to p4 labelsync in VCP::Des::P4, reported by david d zuhn <zoo@bravara.com> |
||
#8 | 669 | barrie_slaymaker |
0.1 Wed Jul 4 00:27:35 EDT 2001 Fix VCP::Dest::p4 to take the filespec from the p4:<dest> spec and use it as the rev_root. No --rev-root option at this time, not sure if it's needed. Reported by david d zuhn <zoo@bravara.com>. |
||
#7 | 628 | barrie_slaymaker | Cleaned up POD in bin/vcp, added BSD-style license. | ||
#6 | 609 | barrie_slaymaker |
Add a file to the test procedure that it alternately added and deleted (file is named "readd"). Fixed all destinations to handle that. |
||
#5 | 608 | barrie_slaymaker |
Lots of changes to get vcp to install better, now up to 0.066. Many thanks to Matthew Attaway for testing & suggestions. |
||
#4 | 480 | barrie_slaymaker |
0.06 Wed Dec 20 23:19:15 EST 2000 - bin/vcp: Added --versions, which loads all modules and checks them for a $VERSION and print the results out. This should help with diagnosing out-of-sync modules. - Added $VERSION vars to a few modules :-). Forgot to increment any $VERSION strings. - VCP::Dest::cvs: The directory "deeply" was not being `cvs add`ed on paths like "a/deeply/nested/file", assuming "deeply" had no files in it. - VCP::Dest::revml: fixed a bug that was causing files with a lot of linefeeds to be emitted in base64 instead of deltaed. This means most text files. - Various minor cleanups of diagnostics and error messages, including exposing "Can't locate Foo.pm" when a VCP::Source or VCP::Dest module depends on a module that's not installed, as reported by Jeff Anton. |
||
#3 | 478 | barrie_slaymaker |
0.05 Mon Dec 18 07:27:53 EST 2000 - Use `p4 labels //...@label` command as per Rober Cowham's suggestion, with the '-s' flag recommended by Christopher Siewald and Amaury.FORGEOTDARC@atsm.fr. Though it's actually something like vcp: running /usr/bin/p4 -u safari -c safari -p localhost:5666 -s files //.../NtLkly //...@compiler_a3 //.../NtLkly //...@compiler_may3 and so //on //for 50 parameters to get the speed up. I use the //.../NtLkly "file" as //a separator between the lists of files in various //revisions. Hope nobody has any files named that :-). What I should do is choose a random label that doesn't occur in the labels list, I guess. - VCP::Source::revml and VCP::Dest::revml are now binary, control code, and "hibit ASCII" (I know, that's an oxymoron) clean. The <comment>, <delta>, and <content> elements now escape anything other than tab, line feed, space, or printable chars (32 <= c <= ASCII 126) using a tag like '<char code="0x09">'. The test suite tests all this. Filenames should also be escaped this way, but I didn't get to that. - The decision whether to do deltas or encode the content in base64 is now based on how many characters would need to be escaped. - We now depend on the users' diff program to have a "-a" option to force it to diff even if the files look binary to it. I need to use Diff.pm and adapt it for use on binary data. - VCP::Dest::cvs now makes sure that no two consecutive revisions of the same file have the same mod_time. VCP::Source::p4 got so fast at pulling revisions from the repositories the test suite sets up that CVS was not noticing that files had changed. - VCP::Plugin now allows you to set a list of acceptable result codes, since we now use p4 in ways that make it return non-zero result codes. - VCP::Revs now croaks if you try to add two entries of the same VCP::Rev (ie matching filename and rev_id). - The <type> tag is now limited to "text" or "binary", and is meant to pass that level of info between foreign repositories. - The <p4_info> on each file now carries the one line p4 description of the file so that p4->p4 transferes can pick out the more detailed info. VCP::Source::p4, VCP::Dest::p4 do this. - VCP::{Source,Dest}::{p4,cvs} now set binaryness on added files properly, I think. For p4->p4, the native p4 type is preserved. For CVS sources, seeing the keyword substitution flag 'o' or 'b' implies binaryness, for p4, seeing a filetype like qr/u?x?binary/ or qr/x?tempobj/ or "resource" implies binaryness (to non-p4 destinations). NOTE: Seeing a 'o' or 'b' in a CVS source only ends up setting the 'b' option on the destination. That should be ok for most uses, but we can make it smarter for cvs->cvs transfers if need be. |
||
#2 | 468 | barrie_slaymaker |
- VCP::Dest::p4 now does change number aggregation based on the comment field changing or whenever a new revision of a file with unsubmitted changes shows up on the input stream. Since revisions of files are normally sorted in time order, this should work in a number of cases. I'm sure we'll need to generalize it, perhaps with a time thresholding function. - t/90cvs.t now tests cvs->p4 replication. - VCP::Dest::p4 now doesn't try to `p4 submit` when no changes are pending. - VCP::Rev now prevents the same label from being applied twice to a revision. This was occuring because the "r_1"-style label that gets added to a target revision by VCP::Dest::p4 could duplicate a label "r_1" that happened to already be on a revision. - Added t/00rev.t, the beginnings of a test suite for VCP::Rev. - Tweaked bin/gentrevml to comment revisions with their change number instead of using a unique comment for every revision for non-p4 t/test-*-in-0.revml files. This was necessary to test cvs->p4 functionality. |
||
#1 | 467 | barrie_slaymaker | Version 0.01, initial checkin in perforce public depot. |