package VCP::Dest::p4 ; =head1 NAME VCP::Dest::p4 - p4 destination driver =head1 SYNOPSIS vcp <source> p4:user:password@p4port:<dest> vcp <source> p4:user(client):password@p4port:<dest> vcp <source> p4:<dest> =head1 DESCRIPTION The <dest> spec is a perforce repository spec and must begin with // and a depot name ("//depot"), not a local filesystem spec or a client spec. There should be a trailing "/..." specified. If no user name, password, or port are given, the underlying p4 command will look at that standard environment variables. VCP sets the environment P4PASSWD rather than giving p4 the password on the command line so it won't be logged in debugging or error messages. The other options are passed on the command line. If no client name is given, a temporary client name like "vcp_tmp_1234" will be created and used. The P4CLIENT environment variable will not be used. If an existing client name is given, the named client spec will be saved off, altered, used, and restored. If the client was created for this import, it will be deleted when complete, regardless of whether the client was specified by the user or was randomly generated. WARNING: If perl coredumps or is killed with a signal that prevents cleanup--like a SIGKILL (9)--the the client deletion or restoral will not occur. The client view is not saved on disk, either, so back it up manually if you care. THE CLIENT SAVE/RESTORE FEATURE IS EXPERIMENTAL AND MAY CHANGE BASED ON USER FEEDBACK. VCP::Dest::p4 attempts change set aggregation by sorting incoming revisions. See L<VCP::Dest/rev_cmp_sub> for the order in which revisions are sorted. Once sorted, a change is submitted whenever the change number (if present) changes, the comment (if present) changes, or a new rev of a file with the same name as a revision that's pending. THIS IS EXPERIMENTAL, PLEASE DOUBLE CHECK EVERYTHING! =head1 OPTIONS =for test_scripts t/90revml2p4_0.t t/90revml2p4_1.t t/90revml2p4_2.t t/90revml2p4_3.t =over =item --run-p4d Runs a p4d instance in the directory indicated by repo_server (use a directory path rather than a host name). If repo_server contains a port, that port will be used, otherwise a random port will be used. Dies unless the directory exists and contains files matching db.* (to help prevent unexpected initializing of empty directories). VCP will kill this p4d when it's done. =item --init-p4d Initializes a directory and starts a p4d in it on the given port. Refuses to init a non-empty directory. In this case the p4port portion of the destination specification must point to a directory; and the port, if present, will be used for the port (otherwise a randomized port number other than p4d's 1666 default will be used.) A temporary p4d will be started that should be shut down upon process exit. If the process does not exit cleanly (for instance, if sent the QUIT signal), this shutdown may not occur. =item --delete-p4d-dir If C<--init-p4d> is passed and the target directory is not empty, it will be removed before running the p4d. THIS IS DANGEROUS AND SHOULD ONLY BE USED IN TEST ENVIRONMENTS. =item --change-branch-rev-1 Some SCMs don't create a branch of a file until there is actually a change made to that file. So the first revision of a file on a branch is different from its parent on the main branch. Normally, p4 does not work this way: rev #1 of a branched file is a very inexpensive copy of the parent revision: you do a p4 integrate, submit, edit, submit sequence to branch a file and introduce it's changes. This option forces VCP to do a p4 integrate, add, submit sequence to branch files, thus capturing the branch and the file alterations in one change. Using this option allows VCP to more exactly model the source repository in the destination repository revision-for-revision, but leaves you with a perforce repository that may not be consistent with your work practices, so it is not the default behavior. =back =head1 NOTES The p4 destination driver allows branching from deleted revisions of files to accomodate source repositories that allow it. It does this by branching from the revision prior to the deleted revision. The p4 destination driver also allows the first revision of a file to be a deleted revision by forcing an empty "add" followed by a "delete". CVS does this on the main trunk (rev 1.1 is "dead", ie deleted) when you add a file on a branch. =cut $VERSION = 1 ; use strict ; use vars qw( $debug ) ; $debug = 0 ; use Carp ; use File::Basename ; use File::Path ; use VCP::Debug ':debug' ; use VCP::Dest ; use VCP::Logger qw( lg lg_fh pr ); use VCP::Rev ; use VCP::Utils qw( empty ); use VCP::Utils::p4 qw( underscorify_name ); ## If we ever want to store state in the dest repo, this constant ## turns that on. It should become an option if it is ever ## reenabled, probably replacing the VCP::RevMapDB. use constant store_state_in_repo => 0; use base qw( VCP::Dest VCP::Utils::p4 ) ; use fields ( # 'P4_SPEC', ## The root of the tree to update 'P4_PENDING', ## Revs pending the next submit 'P4_WORK_DIR', ## Where to do the work. 'P4_REPO_CLIENT', ## See VCP::Utils::p4 for accessors and usage... 'P4_INIT_P4D', ## --init-p4d flag 'P4_DELETE_P4D_DIR', ## --delete_p4d flag (delete temp p4d dir before starting p4d) 'P4_RUN_P4D', ## --run_p4d flag (run temp p4d) 'P4_LABEL_FORM', ## A cached label form 'P4_ADDED_LABELS', ## A hash of labels that we've already added. ## TODO: Preload this using the p4 labels command ## to save some time when writing to big repos? 'P4_CHANGE_BRANCH_REV_1', ## Which command sequence to use when ## branching a file. See POD above. ## members for change number divining: 'P4_PREV_CHANGE_ID', ## The change_id in the r sequence, if any 'P4_PREV_COMMENT', ## Used to detect change boundaries 'P4_REV_MAP', ## RevMapDB ) ; sub new { my $class = shift ; $class = ref $class || $class ; my VCP::Dest::p4 $self = $class->SUPER::new( @_ ) ; ## Parse the options my ( $spec, $options ) = @_ ; $self->parse_p4_repo_spec( $spec ) ; $self->repo_id( "p4:" . $self->repo_server ); $self->parse_options( $options, "init-p4d!" => \$self->{P4_INIT_P4D}, "change-branch-rev-1!" => \$self->{P4_CHANGE_BRANCH_REV_1}, "delete-p4d-dir!" => \$self->{P4_DELETE_P4D_DIR}, "run-p4d!" => \$self->{P4_RUN_P4D}, ); return $self ; } sub init { my VCP::Dest::p4 $self = shift ; ## We use the rev_root only to munge branch specs. ## We let p4 set the rev_root by setting the ## client view to the destination path the user specified, so as perforce ## adds each file in our working dir, it puts them in the right spot ## in the repository (under the destination rev_root). $self->deduce_rev_root( $self->repo_filespec ); if ( $self->{P4_INIT_P4D} ) { if ( $self->{P4_DELETE_P4D_DIR} ) { $self->rev_map->delete_db; $self->head_revs->delete_db; $self->files->delete_db; } $self->init_p4d; } else { pr "ignoring --delete-p4d-dir, which is only useful with --init-p4d\n" if $self->{P4_DELETE_P4D_DIR}; $self->run_p4d if $self->{P4_RUN_P4D}; } $self->rev_map->open_db; $self->head_revs->open_db; $self->files->open_db; $self->set_up_p4_user_and_client; $self->init_p4_view ; } sub init_p4d { my $self = shift; my ( $dir, $port ) = split ":", $self->repo_server, 2; my @files; @files = glob "$dir/*" if -d $dir; if ( @files && $self->{P4_DELETE_P4D_DIR} ) { require File::Path; rmtree [ @files ]; @files = glob "$dir/*"; } die "cannot --init-p4d on non-empty dir $dir\n" if @files; $self->mkdir( $dir ) unless -e $dir; ## VCPP4LICENSE env var, if present, points to a p4 license file to ## use. Link to it. my $license = $ENV{VCPP4LICENSE} ; if ( ! empty $license && -r $license && ! -f "$dir/license" ) { symlink $license, "$dir/license" or die "failed to link '$dir/license' to '$license' (p4 license)"; pr "linked '$dir/license' to '$license' (p4 license)\n"; } ## Ok, this is wierd: we need to fork & run p4d in foreground mode ## so that we can capture it's PID and kill it later. There ## doesn't seem to be the equivalent of a 'p4d.pid' file. If we let ## it daemonize, then I don't know how to get it's PID. $port = $self->launch_p4d( $dir, $port ); $self->repo_server( "localhost:$port" ); } sub checkout_file { my VCP::Dest::p4 $self = shift ; my VCP::Rev $r ; ( $r ) = @_ ; confess unless defined $self && defined $self->header ; debug "retrieving '", $r->as_string, "' from p4 dest repo" if debugging; ## The rev_root was put in the client view, p4 will "denormalize" ## the name for us. my $work_path = $self->work_path( $r->name ) ; debug "work_path '$work_path'" if debugging; my ( undef, $work_dir ) = fileparse( $work_path ) ; $self->mkpdir( $work_path ) unless -d $work_dir ; my $tag = store_state_in_repo ? "\@vcp_" . underscorify_name $r->id : "#" . ($self->rev_map->get( [ $r->source_repo_id, $r->id ] ))[1]; ## The -f forces p4 to sync even if it thinks it doesn't have to. It's ## not in there for any known reason, just being conservative. $self->p4( ['sync', '-f', $r->name . $tag ] ) ; die "'$work_path' not created in backfill" unless -e $work_path ; return $work_path ; } sub handle_header { my VCP::Dest::p4 $self = shift ; my ( $h ) = @_; $self->{P4_PENDING} = [] ; $self->{P4_PREV_COMMENT} = undef ; $self->{P4_PREV_CHANGE_ID} = undef ; $self->{P4_LABEL_FORM} = undef ; $self->{P4_ADDED_LABELS} = {}; $self->SUPER::handle_header( @_ ) ; if ( $h->{branches} ) { for my $b ( $h->{branches}->get ) { my $spec = $b->p4_branch_spec; next if empty $spec; ## Re-root the view. my $found_it; $spec = $self->build_p4_form( map { if ( $found_it ) { ( my $source_root = $h->{ rev_root} ) =~ s{^/*}{//}; ( my $dest_root = $self->rev_root ) =~ s{^/*}{//}; s{\Q$source_root}{$dest_root}g; undef $found_it; } $found_it = $_ eq "View"; $_; } $self->parse_p4_form( $spec ) ); $self->p4( [qw(branch -i -f)], \$spec ); } } } sub handle_rev { my VCP::Dest::p4 $self = shift ; my VCP::Rev $r ; ( $r ) = @_ ; debug "handle_rev got ", $r->as_string if debugging; my $change_id = $r->change_id; if ( @{$self->{P4_PENDING}} && ! $self->{P4_CHANGE_BRANCH_REV_1} ? $change_id ne $self->{P4_PREV_CHANGE_ID} && ( debugging() ? debug "time to submit: change_id changed" : 1 ) : ( defined $r->comment && defined $self->{P4_PREV_COMMENT} && $r->comment ne $self->{P4_PREV_COMMENT} && ( debugging() ? debug "time to submit: comment changed [", $r->comment, "] vs [", $self->{P4_PREV_COMMENT}, "]" : 1 ) ) || ( ! $r->is_placeholder_rev && grep( ! $_->is_placeholder_rev && $r->name eq $_->name, @{$self->{P4_PENDING}} ) && ( debugging() ? debug "time to submit: name repeated" : 1 ) ) || ( $r->previous && grep( ! $_->is_placeholder_rev && $_ == $r->previous, @{$self->{P4_PENDING}} ) && ( debugging() ? debug "time to submit: need a previous revision for ", "\n", " ", $r->as_string, "\n", map " " . $_->as_string . "\n", grep( $_ == $r->previous, @{$self->{P4_PENDING}} ) : 1 ) ) ) { $self->submit; $self->{P4_PREV_COMMENT} = $r->comment; } $self->{P4_PREV_CHANGE_ID} = $change_id; ## TODO: batch these too so that we can pass them all to ## fetch_files() and allow the source to perhaps operate ## more efficiently. if ( $r->is_base_rev ) { my ( $work_path ) = VCP::Revs->fetch_files( $r ); $self->compare_base_revs( $r, $work_path ); return; } push @{$self->{P4_PENDING}}, $r; } sub _add_it { my VCP::Dest::p4 $self = shift; my VCP::Rev $r ; ( $r ) = @_ ; my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; ! -e $work_path; } sub handle_footer { my VCP::Dest::p4 $self = shift ; $self->submit if @{$self->{P4_PENDING}}; $self->SUPER::handle_footer ; } { my $lg_fh = lg_fh; my $change_spec; sub p4_submit { my VCP::Dest::p4 $self = shift; my ( $max_time, $description, $is_integrate, @revs ) = @_; unless ( $change_spec ) { $self->p4( [ "change", "-o" ], undef, \$change_spec ) ; ## ASSume the Files & Description are the last two fields in ## the spec. $change_spec =~ s/^(Description|Files):.*\r?\n\r?.*//ms or die "couldn't remove change file list and description\n$change_spec" ; } my @revs_to_submit = grep $is_integrate ? $_->is_placeholder_rev : ! $_->is_placeholder_rev, @revs; if ( length $description ) { $description =~ s/^/\t/gm ; chomp $description; } my $files = join "", map "\t//" . $self->denormalize_name( $_->name ) . "\n", @revs_to_submit; my $change = $change_spec . <<END_SPEC; Description: $description Files: $files END_SPEC my $submit_log; $self->p4([ "submit", "-i"], \$change, \$submit_log ) ; ## extract the change number and the file list from the submit output print $lg_fh $submit_log; my $change_number; my %p4_rev_ids; { while ( $submit_log =~ m{\G(.*)([\r\n]+|\z)}g ) { my $line = $1; if ( $line =~ m{^\w+\s+//(.*)#(\d+)\z} ) { $p4_rev_ids{$1} = $2; } elsif ( $line =~ m{^Change (\d+) } ) { $change_number = $1; } } } pr "p4 change $change_number submitted" if defined $change_number; ## Force the correct date and a user id { my $repl = "Change:\t$change_number\n\n"; $repl .= "Date:\t$max_time\n\n" if defined $max_time; $change =~ s/^Change:.*\r?\n/$repl/m or die "couldn't modify change number:\n$change"; $change =~ s/^Status:.*\r?\n/Status: submitted\n\n/m or die "couldn't modify status:\n$change"; ## ASSume the files list is last $change =~ s/^Files:.*\r?\n//ms or die "couldn't remove files list:\n$change"; my $user_id; for ( @{$self->{P4_PENDING}} ) { unless ( empty $_->user_id ) { $user_id = underscorify_name( $_->user_id ); last; } } if ( defined $user_id ) { $change =~ s/^User:.*/User:\t$user_id/m or die "couldn't modify change user\n$change" ; } $self->p4( [ "change", "-i", "-f" ], \$change ) ; } for my $r ( @revs_to_submit ) { my $denorm_name = $self->denormalize_name( $r->name ); my $rev_id = $p4_rev_ids{$denorm_name}; unless ( defined $rev_id ) { $submit_log =~ s/^/ /mg; require Data::Dumper; die "no rev number found in p4 submit output for ", $r->as_string, ":\n", $submit_log, "looked for: $denorm_name\n", "p4 revs parsed: ", Data::Dumper::Dumper( \%p4_rev_ids ); } $self->rev_map->set( [ $r->source_repo_id, $r->id ], $denorm_name, $rev_id, $self->files->get( [ $denorm_name ] ), ); $self->head_revs->set( [ $r->source_repo_id, $r->source_filebranch_id ], $r->source_rev_id ); } } } my $branch_spec; sub _integrate_pending { my VCP::Dest::p4 $self = shift ; my @to_integrate; my @to_resolve; my @integrated; my $max_time; for my $r ( @{$self->{P4_PENDING}} ) { my $pr = $r->previous; if ( $pr ## && $self->_add_it( $r ) && ( $pr->branch_id || "" ) ne ( $r->branch_id || "" ) ) { ## Branch the previous version to make the new one. Leave ## add_it set so we can drop the new one in over the ## branched version in case it's changed. my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; my $denorm_name = $self->denormalize_name( $fn ); my ( $pfull_name, $prev_id, $pstate ) = $self->rev_map->get( [ $pr->source_repo_id, $pr->id ] ); die "Can't integrate from unknown revision:\n", "parent: ", $pr->as_string, "\n", "child: ", $r->as_string, "\n" unless defined $pstate; if ( $pstate eq "deleted" ) { ## CVS allows branching from "deleted" revisions, ## let's hope that the revision before the previous ## revision is *not* deleted. $prev_id--; } push @to_integrate, [ $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id, ]; $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; } } return unless @to_integrate; unless ( $branch_spec ) { $self->p4( [ "branch", "-o", "vcp_$$" ], \undef, \$branch_spec ); $branch_spec =~ s/^(View:).*/$1\n/ms; } my $this_branch_spec = join "", $branch_spec, map { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$_; qq{\t "//$pfull_name" "//$denorm_name"\n}; } @to_integrate; $self->p4( [ "branch", "-i" ], \$this_branch_spec ); my $to_integrate = join "", map { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$_; "//$denorm_name#$prev_id\n" } @to_integrate; $self->p4_x( [ "integrate", "-b", "vcp_$$" ], \$to_integrate ); for my $i ( @to_integrate ) { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$i; # $self->p4( [ "integrate", "//$pfull_name#$prev_id", $fn ] ) ; push @to_resolve, $fn if $self->files->exists( [ $denorm_name ] ); $self->files->set( [ $denorm_name ], "integrated to" ); if ( $self->{P4_CHANGE_BRANCH_REV_1} ) { ## Force it to be `p4 add`ed as well by _add_edit_delete() debug "unlinking $work_path so it will be added\n" if debugging; unlink $work_path or die "$! unlinking $work_path\n" if -e $work_path; } else { ## Integrate the first change push @integrated, $i; } } $self->p4( [ "resolve", "-at" ] ) if @to_resolve; return unless @integrated; $self->p4_submit( $max_time, "create branches", "is integrate", map $_->[0], @integrated ); for my $i ( @integrated ) { my ( $r, $fn, $denorm_name, $work_path, $pfull_name, $prev_id ) = @$i; lg "syncing recently integrated file $fn"; $self->p4( [ "sync", "$fn" ] ) unless -e $work_path; } } sub _pre_add_pending_initial_deletes { my VCP::Dest::p4 $self = shift ; my @to_add; my $max_time; for my VCP::Rev $r ( @{$self->{P4_PENDING}} ) { next if $r->is_placeholder_rev; if ( $r->action eq "delete" ) { my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; my $denorm_name = $self->denormalize_name( $fn ); my ( @state ) = $self->files->get( [ $denorm_name ] ); unless ( @state ) { ## Never seen this file before. ## ## This could be a fake deletion from CVS, which does this when ## a file is added on a branch. So we fake up an empty ## file and add it. lg "creating '$work_path' so that it may be deleted"; $self->mkpdir( $work_path ); open F, ">$work_path" or die "$! opening $work_path\n"; close F; $r->dest_work_path( $work_path ) ; push @to_add, [ $r, $denorm_name ]; $self->files->set( [ $denorm_name ], "touched and added" ); $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; } } } if ( @to_add ) { $self->p4_x( [ "add" ], [ map $_->[0]->name . "\n", @to_add ] ); $self->p4_submit( $max_time, "spoof addition of deleted file", 0, map $_->[0], @to_add ); } } ## Revs are only queued up by handle_rev() because we need to ## deal with integrate commands before we deal with other commands. sub _add_edit_delete_pending { my VCP::Dest::p4 $self = shift; my $do_submit; my @to_delete; my %to_add; ## Keyed on filetype my %to_edit; ## Keyed on filetype my @to_link; for my VCP::Rev $r ( @{$self->{P4_PENDING}} ) { next if $r->is_placeholder_rev; ++$do_submit; my $fn = $r->name ; my $work_path = $self->work_path( $fn ) ; debug "work_path '$work_path'" if debugging; my $denorm_name = $self->denormalize_name( $r->name ); if ( $r->action eq "delete" ) { my ( @state ) = $self->files->get( [ $denorm_name ] ); my $deleteit = $state[0] ne "deleted"; if ( $deleteit && ! -e $work_path ) { $self->p4( [ "sync", "-f", $fn ] ); } if ( -e $work_path ) { unlink $work_path or die "$! unlinking $work_path" ; } push @to_delete, $fn if $deleteit; $self->files->set( [ $denorm_name ], "deleted" ); } else { ## add or edit my $filetype = defined $r->p4_info && $r->p4_info =~ /\((\S+)\)$/ ? $1 : $r->type ; if ( $self->_add_it( $r ) ) { $self->mkpdir( $work_path ) ; push @{$to_add{$filetype}}, $r; $self->files->set( [ $denorm_name ], "added" ); } else { push @{$to_edit{$filetype}}, $r; $self->files->set( [ $denorm_name ], "edited" ); } my $work_path = $self->work_path( $r->name ) ; $r->dest_work_path( $work_path ) ; push @to_link, $r; ## TODO: Provide command line options for user-defined tag prefixes $r->add_label( "vcp_" . $r->id ) if store_state_in_repo; } } $self->p4_x( [ "delete" ], [ map "$_\n", @to_delete ] ) if @to_delete; for ( keys %to_edit ) { my @to_edit = @{$to_edit{$_}}; $self->p4_x( [ "edit", "-t", $_ ], [ map $_->name . "\n", @to_edit ] ); for my $r ( @to_edit ) { my $work_path = $self->work_path( $r->name ) ; unlink $self->work_path( $r->name ) or die "$! unlinking $work_path" } } my @source_fns = VCP::Revs->fetch_files( @to_link ); for my $r ( @to_link ) { my $source_fn = shift @source_fns; my $work_path = $r->dest_work_path; if ( $source_fn ne $work_path ) { debug "linking $source_fn to $work_path" if debugging; link $source_fn, $work_path or die "$! linking '$source_fn' -> '$work_path'" ; } if ( defined $r->mod_time ) { utime $r->mod_time, $r->mod_time, $work_path or die "$! changing times on $work_path" ; } } $self->p4_x( [ "add", "-t", $_ ], [ map $_->name . "\n", @{$to_add{$_}} ] ) for keys %to_add; return $do_submit; } sub submit { my VCP::Dest::p4 $self = shift ; ## Take care of any integrations up front so that they're ## present if the first rev on a branch has changes on it so it ## can be "p4 edit"ed in the main submit. $self->_integrate_pending; ## CVS can create initial revs that are deleted; p4 must add them ## then delete them. $self->_pre_add_pending_initial_deletes( $_ ); my $things_to_submit = $self->_add_edit_delete_pending; if ( $things_to_submit ) { my %pending_labels ; my %comments ; my $max_time ; if ( @{$self->{P4_PENDING}} ) { for my $r ( @{$self->{P4_PENDING}} ) { $comments{$r->comment} = $r->name if defined $r->comment ; $max_time = $r->time if ! defined $max_time || $r->time > $max_time ; for my $l ( $r->labels ) { push @{$pending_labels{$l}}, $r->dest_work_path ; } } if ( defined $max_time ) { my @f = reverse( (localtime $max_time)[0..5] ) ; $f[0] += 1900 ; ++$f[1] ; ## Day of month needs to be 1..12 $max_time = sprintf "%04d/%02d/%02d %02d:%02d:%02d", @f ; } elsif ( debugging ) { debug "No max_time found" ; } } my $description = join( "\n", keys %comments ) ; $self->p4_submit( $max_time, $description, 0, @{$self->{P4_PENDING}} ); ## Create or add a label spec for each of the labels. The 'sort' is to ## make debugging output more legible. ## TODO: Modify RevML to allow label metadata (owner, desc, options) ## to be passed through. Same for user, client, jobs metadata etc. ## The assumption is made that most labels will apply to a single change ## number, so we do the labelling once per submit. I don't think that ## this will break if it doesn't, but TODO: add more labelling tests. pr "storing " . keys( %pending_labels ) . " labels" if keys %pending_labels > 25; for my $label ( sort keys %pending_labels ) { my $p4_label = underscorify_name $label; $self->{P4_ADDED_LABELS}->{$p4_label} ||= do { $self->{P4_LABEL_FORM} ||= do { my $out; $self->p4( [qw( label -o ), $p4_label], undef, \$out ) ; $out =~ s/(^.+:\s+)\Q$p4_label\E$/$1<<<LABEL>>>/gm; $out; }; ( my $l_desc = $self->{P4_LABEL_FORM} ) =~ s/<<<LABEL>>>/$p4_label/g; $self->p4( [qw( label -i ) ], \$l_desc, \my $dev_null ) ; 1; }; my $files_to_label = join( "\n", @{$pending_labels{$label}} ) . "\n" ; $self->p4_x( [qw( labelsync -a -l ), $p4_label ], \$files_to_label, \my $dev_null ) ; } } ## Make GC and file cleanup occur ASAP for ( @{$self->{P4_PENDING}} ) { $_->previous( undef ); $_->set_work_path( undef ) if defined $_->work_path; } @{$self->{P4_PENDING}} = () ; } ## Prevent VCP::Plugin from rmtree-ing the workspace we're borrowing sub DESTROY { my VCP::Dest::p4 $self = shift ; $self->work_root( undef ) ; $self->SUPER::DESTROY ; } =head1 AUTHOR Barrie Slaymaker <barries@slaysys.com> =head1 COPYRIGHT Copyright (c) 2000, 2001, 2002 Perforce Software, Inc. All rights reserved. See L<VCP::License|VCP::License> (C<vcp help license>) for the terms of use. =cut 1
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#129 | 5402 | Barrie Slaymaker | - Disallow "?" in p4 names | ||
#128 | 4521 | Barrie Slaymaker | - Debugging lg() calls removed | ||
#127 | 4520 | Barrie Slaymaker |
- VCP::Dest::p4 another "No file(s) to resolve" error squashed - This occured when cloning or branching a file over a previously deleted file |
||
#126 | 4507 | Barrie Slaymaker |
- RevML: - added <action>, removed <delete>, <placeholder> and <move> - added <from_id> for clones (and eventually merge actions) - Simplified DTD (can't branch DTD based on which action any more) - VCP::Source::cvs, VCP::Filter::changesets and VCP::Dest::p4 support from_id in <action>clone</action> records - VCP::Dest::perl_data added - VCP::Rev::action() "branch" added, no more undefined action strings - "placeholder" action removed |
||
#125 | 4489 | Barrie Slaymaker | - removed commented-out debug code | ||
#124 | 4476 | Barrie Slaymaker | - misc bugfixes | ||
#123 | 4416 | Barrie Slaymaker | - VCP::Dest::p4 tries not to send pre-epoch timestamps to p4 | ||
#122 | 4411 | Barrie Slaymaker | - missing metadata is logged in the log file | ||
#121 | 4166 | Barrie Slaymaker | - No more VCP::Dest::p4 BUG on labelling a freshly branch file | ||
#120 | 4165 | Barrie Slaymaker |
- VCP::Source::null now returns /dev/null or NUL for files - VCP::Dest::p4 now watches is_branch_rev() to detect branches - t/99p4_label_branch_rev_1.t reproduces a bug reported by Matt Attaway < matt at perforce com > |
||
#119 | 4033 | Barrie Slaymaker | - VCP::Dest::p4 now really estimates time | ||
#118 | 4032 | Barrie Slaymaker | - VCP::Dest::p4 now estimates missing metadata | ||
#117 | 4021 | Barrie Slaymaker |
- Remove all phashes and all base & fields pragmas - Work around SWASHGET error |
||
#116 | 4012 | Barrie Slaymaker | - Remove dependance on pseudohashes (deprecated Perl feature) | ||
#115 | 4006 | Barrie Slaymaker | - VCP will now use the P4::Client module if it's installed. | ||
#114 | 4003 | Barrie Slaymaker |
- VCP::Dest::p4 now prevents illegal filenames, user_ids, and label names |
||
#113 | 3997 | Barrie Slaymaker | - Limit p4 integrate size to 100 files | ||
#112 | 3975 | Barrie Slaymaker |
- VCP::Dest::p4 can now branch the same file multiple times in one change. |
||
#111 | 3970 | Barrie Slaymaker |
- VCP::Source handles rev queing, uses disk to reduce RAM - Lots of other fixes |
||
#110 | 3930 | Barrie Slaymaker |
- VCP::Source::cvs and VCP::Dest::p4 handle cloning deletes - "placeholder" actions and is_placeholder_rev() deprecated in favor of is_branch_rev() and is_clone_rev(). - Misc cleanups and minor bugfixes |
||
#109 | 3916 | Barrie Slaymaker | - Reduce memory consumption | ||
#108 | 3907 | Barrie Slaymaker | - Debugging cleanups | ||
#107 | 3903 | Barrie Slaymaker | - Minor debug logging impovements | ||
#106 | 3893 | Barrie Slaymaker | - p4d is now launched in the proper directory | ||
#105 | 3855 | Barrie Slaymaker |
- vcp scan, filter, transfer basically functional - Need more work in re: storage format, etc, but functional |
||
#104 | 3850 | Barrie Slaymaker | - No longer stores all revs in memory | ||
#103 | 3837 | Barrie Slaymaker | - Improved progress bar support | ||
#102 | 3813 | Barrie Slaymaker | - VCP::Rev::previous() is no more | ||
#101 | 3812 | Barrie Slaymaker | - VCP::Dest::* no longer need VCP::Rev->previous() | ||
#100 | 3811 | Barrie Slaymaker | - fetch_*() and get_rev() renamed get_source_file() | ||
#99 | 3805 | Barrie Slaymaker | - VCP::Revs::fetch_files() removed | ||
#98 | 3754 | Barrie Slaymaker |
- VCP::Dest::p4 no longer tries to label files the current change just deleted. |
||
#97 | 3749 | Barrie Slaymaker | - error message tweaked | ||
#96 | 3748 | Barrie Slaymaker | - VCP::Dest::p4 dies when trying to branch to/from same file | ||
#95 | 3741 | Barrie Slaymaker | - Minor message cleanup | ||
#94 | 3712 | Barrie Slaymaker | - Bogus confess() doesn't bite any more | ||
#93 | 3706 | Barrie Slaymaker | - VCP gives some indication of output progress (need more) | ||
#92 | 3647 | Barrie Slaymaker |
- All UI prompts & descriptions rewritten. - Minor tweak to VCP::Dest::p4 P4USER defaulting |
||
#91 | 3572 | John Fetkovich | added y/n question to accept default of user_id | ||
#90 | 3569 | Barrie Slaymaker |
- Work around bug caused by p4's using the long pathname when $ENV{PWD} is not set. |
||
#89 | 3518 | John Fetkovich | more interactive ui improvements | ||
#88 | 3498 | Barrie Slaymaker | - Fix changeset boundary detection bug in VCP::Dest::p4 | ||
#87 | 3491 | Barrie Slaymaker | - All sections are now documented in generated config files | ||
#86 | 3460 | Barrie Slaymaker |
- Revamp Plugin/Source/Dest hierarchy to allow for reguritating options in to .vcp files |
||
#85 | 3442 | Barrie Slaymaker | - Win32 regex fixup for label specs | ||
#84 | 3432 | Barrie Slaymaker | - "p4 print" to "p4 sync" | ||
#83 | 3423 | Barrie Slaymaker | - Use the new VCP::Utils::p4::split_repo_server() | ||
#82 | 3411 | Barrie Slaymaker |
- p4 output parsing improvment - clearer log messages |
||
#81 | 3402 | Barrie Slaymaker |
- now passes all tests using the p4 api library. (still not default, set env var VCPP4API=1) - foo->p4 handles branch-but-no-change case when --change-branch-rev-1 is passed. - sources & dests can now provide their own command execution routine in place of shelling out to an external command (as in call the p4api library instead of running the p4 command). |
||
#80 | 3382 | John Fetkovich |
Moved setting of repo_id to 'sub init' moved defaulting of repo_server to P4PORT env var to 'sub init' |
||
#79 | 3284 | John Fetkovich |
'sub new' constructor in Source and Dest p4.pm fixed so parse_p4_repo_spec only called when a $spec is provided to the constructor. parse_p4_repo_spec now also sets the repo_id. parse_repo_spec (TODO item) no longer returns a hash value of the values parsed, it only sets fields in $self. Fixed a few places where that return hash was used. |
||
#78 | 3277 | John Fetkovich | split out 'sub init' from 'sub new' | ||
#77 | 3207 | John Fetkovich | doc (pod) fixes | ||
#76 | 3187 | Barrie Slaymaker |
Tweak p4 change -o replacement logic to work with command line client. |
||
#75 | 3186 | Barrie Slaymaker | cleanup | ||
#74 | 3185 | Barrie Slaymaker | Reduce number of p4 changes commands to 1 per p4 submit. | ||
#73 | 3169 | Barrie Slaymaker |
Log the stdout of all p4 submits use -x - for p4 integrate calls. Avoid Carp; use BUG. |
||
#72 | 3155 | Barrie Slaymaker |
Convert to logging using VCP::Logger to reduce stdout/err spew. Simplify & speed up debugging quite a bit. Provide more verbose information in logs. Print to STDERR progress reports to keep users from wondering what's going on. Breaks test; halfway through upgrading run3() to an inline function for speed and for VCP specific features. |
||
#71 | 3139 | Barrie Slaymaker | Batch calls to p4 integrate | ||
#70 | 3133 | Barrie Slaymaker |
Make destinations call back to sources to check out files to simplify the architecture (is_metadata_only() no longer needed) and make it more optimizable (checkouts can be batched). |
||
#69 | 3127 | Barrie Slaymaker | Minor cleanups. | ||
#68 | 3120 | Barrie Slaymaker | Move changeset aggregation in to its own filter. | ||
#67 | 3117 | Barrie Slaymaker |
Cut over to faster VCP::Rev::new, remove symbolic method calls. |
||
#66 | 3104 | John Fetkovich |
Added handling of VCPP4LICENSE environment variable in Dest::p4.pm. If that's present and pointing to a readable file when a p4 daemon is started, a symlink will be created in the (possibly newly created) p4root directory to point to the p4 license file pointed to by VCPP4LICENSE. The 'make' target test_all_p4_versions will cycle through each version of p4 and p4d contained in the 'p4versions' directory, in both unlicensed and (if VCPP4LICENSE present) licensed mode. |
||
#65 | 3098 | Barrie Slaymaker |
Convert all length p4 command line calls to us p4 -x -. All hail p4 -x -. |
||
#64 | 3081 | Barrie Slaymaker |
Get cvs->p4 propogation branches with multiple tags working to spec. |
||
#63 | 3076 | Barrie Slaymaker | Improve change aggregation | ||
#62 | 3066 | Barrie Slaymaker |
Set time correctly on integrate and pre-delete add changes. |
||
#61 | 3031 | Barrie Slaymaker |
Fix empty branch creation batch adds, edits, and deletes rearrange for code maintainability |
||
#60 | 3029 | Barrie Slaymaker |
Improve state database (does not affect operation, but was causing some interesting information to be missing when using bin/dump_* |
||
#59 | 3021 | Barrie Slaymaker |
Force integrate to branch from proper revision and allow it to branch from the predecessor of a deleted file (CVS allows file deletes). |
||
#58 | 3011 | Barrie Slaymaker | Free memory and delete files more aggressively | ||
#57 | 2981 | Barrie Slaymaker | quash some output | ||
#56 | 2980 | Barrie Slaymaker |
Don't let placeholders count as a needed previous revision when splitting the rev stream in to changes. |
||
#55 | 2976 | Barrie Slaymaker |
Force submittal of parent revision if a child rev of it arrives before it is submitted. |
||
#54 | 2973 | Barrie Slaymaker | Fix handling of branched but unchanged files | ||
#53 | 2972 | Barrie Slaymaker | Interim checkin | ||
#52 | 2930 | John Fetkovich | added empty() calls | ||
#51 | 2926 | John Fetkovich |
remove --state-location switch add --db-dir and --repo-id switches build state location from concatenation of those two. |
||
#50 | 2915 | Barrie Slaymaker |
Default to *not* changing the first rev of a branch, add --change-branch-rev-1 to enable the non-default behavior (which is what is used by most of the test suite). |
||
#49 | 2913 | Barrie Slaymaker | Put a user id on each change | ||
#48 | 2912 | Barrie Slaymaker | Remove spurious warn()ing | ||
#47 | 2910 | Barrie Slaymaker | Update the submit time of each change. | ||
#46 | 2908 | Barrie Slaymaker | Suppress "refreshed" messages | ||
#45 | 2907 | Barrie Slaymaker |
Use persistent state file to track what's been seen so that delete/add/edit decisions work accross incremental imports |
||
#44 | 2838 | John Fetkovich | Use parse_options rather than using Getopt::Long directly. | ||
#43 | 2802 | John Fetkovich |
Added a source_repo_id to each revision, and repo_id to each Source and Dest. The repo_ids include repository type (cvs,p4,revml,vss,...) and the repo_server fields. Changed the $self->...->set() and $self->...->get() lines in VCP::Dest::* to pass in a conglomerated key value, by passing in the key as an ARRAY ref. Also various restructuring in VCP::DB.pm, VCP::DB_file.pm and VCP::DB_file::sdbm.pm related to this change. |
||
#42 | 2774 | Barrie Slaymaker | Update HeadRevDB on submit/commit/write | ||
#41 | 2770 | Barrie Slaymaker | tweak dump utils: sort output for readability | ||
#40 | 2721 | Barrie Slaymaker | VCP::Dest::p4 now uses RevMapDB | ||
#39 | 2698 | Barrie Slaymaker |
Don't add r_ and ch_ labels, switch to underscorify_name() to get standardized escaping of illegal p4 labels (still need to discover what, exactly, a legal p4 label name is). |
||
#38 | 2664 | Barrie Slaymaker | Try IPC::Run3 to see if it speeds up p4 calls | ||
#37 | 2641 | Barrie Slaymaker |
Add --run-p4d option to VCP::{Source,Dest}::p4. Implement port hunting and p4d up & ready detection for vcp-launched p4ds. |
||
#36 | 2589 | John Fetkovich |
Split 90p4.t into 90revml2p4_0.t, 90revml2p4_1.t, 91p42revml.t, 95p42cvs.t. Added some utilities to the library files listed. |
||
#35 | 2350 | Barrie Slaymaker |
Get p4d to be killed after the last p4 cleanup commands, generalize the p4d launching & destruction system because we'll need it in VCP::Source::p4 too to take checkpoints. |
||
#34 | 2344 | Barrie Slaymaker | Make the labelsync code more self-documenting | ||
#33 | 2336 | Barrie Slaymaker | Notify user when shutting down p4d | ||
#32 | 2333 | Barrie Slaymaker | Tweak logging of p4d command | ||
#31 | 2332 | Barrie Slaymaker | Add --init-p4d and --delete-p4d-dir options | ||
#30 | 2322 | Barrie Slaymaker | Fix jack-in-the-bug options parsing exposed by .vcp files | ||
#29 | 2276 | Barrie Slaymaker |
optimize away extra p4 label -o / -i calls for speed, elegance update t/90p4.t |
||
#28 | 2059 | Barrie Slaymaker | Support for branching in p4->p4 added | ||
#27 | 2049 | Barrie Slaymaker | Get branching working in Dest::p4, clean up some tests. | ||
#26 | 2042 | Barrie Slaymaker | Basic source::p4 branching support | ||
#25 | 2009 | Barrie Slaymaker |
lots of fixes, improve core support for branches and VCP::Source::cvs now supports branches. |
||
#24 | 1855 | Barrie Slaymaker |
Major VSS checkin. Works on Win32 |
||
#23 | 1809 | Barrie Slaymaker | VCP::Patch should ignore lineends | ||
#22 | 1367 | Barrie Slaymaker | lots of docco updates | ||
#21 | 1055 | Barrie Slaymaker |
add sorting, revamp test suite, misc cleanup. Dest/revml is not portable off my system yet (need to release ...::Diff) |
||
#20 | 827 | Barrie Slaymaker | Add a test for and debug p4->cvs incremental exports. | ||
#19 | 825 | Barrie Slaymaker |
test, handle case where no revs are transferred and VCP::Dest::*::handle_footer() blew up. |
||
#18 | 824 | Barrie Slaymaker | Fix p4 backfilling path math to not mangle paths. | ||
#17 | 812 | Barrie Slaymaker |
A more sensible name for on_first_rev(), and allow incremental foo->p4 updates to work by backfilling the base rev instead of trying to add it. |
||
#16 | 723 | Barrie Slaymaker | VCP::Dest::cvs tuning and cvs and p4 bugfixes | ||
#15 | 705 | Barrie Slaymaker | Release 0.22. | ||
#14 | 703 | Barrie Slaymaker | VCP::Source::p4 now uses VCP::Utils::p4::parse_p4_repo_spec() | ||
#13 | 701 | Barrie Slaymaker | Fixed VCP::Dest::p4 re-rooting problem, further t/* cleanup | ||
#12 | 700 | Barrie Slaymaker | Delete orphan code | ||
#11 | 692 | Barrie Slaymaker |
Add VCP::Utils::p4 and use it to get VCP::Dest::p4 to create it's own client view as needed. |
||
#10 | 689 | Barrie Slaymaker |
reinstate -f behavior as the default for VCP::Source::cvs, clean up -D --> -d doco. |
||
#9 | 674 | Barrie Slaymaker |
Fix paths passed to p4 labelsync in VCP::Des::P4, reported by david d zuhn <zoo@bravara.com> |
||
#8 | 669 | Barrie Slaymaker |
0.1 Wed Jul 4 00:27:35 EDT 2001 Fix VCP::Dest::p4 to take the filespec from the p4:<dest> spec and use it as the rev_root. No --rev-root option at this time, not sure if it's needed. Reported by david d zuhn <zoo@bravara.com>. |
||
#7 | 628 | Barrie Slaymaker | Cleaned up POD in bin/vcp, added BSD-style license. | ||
#6 | 609 | Barrie Slaymaker |
Add a file to the test procedure that it alternately added and deleted (file is named "readd"). Fixed all destinations to handle that. |
||
#5 | 608 | Barrie Slaymaker |
Lots of changes to get vcp to install better, now up to 0.066. Many thanks to Matthew Attaway for testing & suggestions. |
||
#4 | 480 | Barrie Slaymaker |
0.06 Wed Dec 20 23:19:15 EST 2000 - bin/vcp: Added --versions, which loads all modules and checks them for a $VERSION and print the results out. This should help with diagnosing out-of-sync modules. - Added $VERSION vars to a few modules :-). Forgot to increment any $VERSION strings. - VCP::Dest::cvs: The directory "deeply" was not being `cvs add`ed on paths like "a/deeply/nested/file", assuming "deeply" had no files in it. - VCP::Dest::revml: fixed a bug that was causing files with a lot of linefeeds to be emitted in base64 instead of deltaed. This means most text files. - Various minor cleanups of diagnostics and error messages, including exposing "Can't locate Foo.pm" when a VCP::Source or VCP::Dest module depends on a module that's not installed, as reported by Jeff Anton. |
||
#3 | 478 | Barrie Slaymaker |
0.05 Mon Dec 18 07:27:53 EST 2000 - Use `p4 labels //...@label` command as per Rober Cowham's suggestion, with the '-s' flag recommended by Christopher Siewald and Amaury.FORGEOTDARC@atsm.fr. Though it's actually something like vcp: running /usr/bin/p4 -u safari -c safari -p localhost:5666 -s files //.../NtLkly //...@compiler_a3 //.../NtLkly //...@compiler_may3 and so //on //for 50 parameters to get the speed up. I use the //.../NtLkly "file" as //a separator between the lists of files in various //revisions. Hope nobody has any files named that :-). What I should do is choose a random label that doesn't occur in the labels list, I guess. - VCP::Source::revml and VCP::Dest::revml are now binary, control code, and "hibit ASCII" (I know, that's an oxymoron) clean. The <comment>, <delta>, and <content> elements now escape anything other than tab, line feed, space, or printable chars (32 <= c <= ASCII 126) using a tag like '<char code="0x09">'. The test suite tests all this. Filenames should also be escaped this way, but I didn't get to that. - The decision whether to do deltas or encode the content in base64 is now based on how many characters would need to be escaped. - We now depend on the users' diff program to have a "-a" option to force it to diff even if the files look binary to it. I need to use Diff.pm and adapt it for use on binary data. - VCP::Dest::cvs now makes sure that no two consecutive revisions of the same file have the same mod_time. VCP::Source::p4 got so fast at pulling revisions from the repositories the test suite sets up that CVS was not noticing that files had changed. - VCP::Plugin now allows you to set a list of acceptable result codes, since we now use p4 in ways that make it return non-zero result codes. - VCP::Revs now croaks if you try to add two entries of the same VCP::Rev (ie matching filename and rev_id). - The <type> tag is now limited to "text" or "binary", and is meant to pass that level of info between foreign repositories. - The <p4_info> on each file now carries the one line p4 description of the file so that p4->p4 transferes can pick out the more detailed info. VCP::Source::p4, VCP::Dest::p4 do this. - VCP::{Source,Dest}::{p4,cvs} now set binaryness on added files properly, I think. For p4->p4, the native p4 type is preserved. For CVS sources, seeing the keyword substitution flag 'o' or 'b' implies binaryness, for p4, seeing a filetype like qr/u?x?binary/ or qr/x?tempobj/ or "resource" implies binaryness (to non-p4 destinations). NOTE: Seeing a 'o' or 'b' in a CVS source only ends up setting the 'b' option on the destination. That should be ok for most uses, but we can make it smarter for cvs->cvs transfers if need be. |
||
#2 | 468 | Barrie Slaymaker |
- VCP::Dest::p4 now does change number aggregation based on the comment field changing or whenever a new revision of a file with unsubmitted changes shows up on the input stream. Since revisions of files are normally sorted in time order, this should work in a number of cases. I'm sure we'll need to generalize it, perhaps with a time thresholding function. - t/90cvs.t now tests cvs->p4 replication. - VCP::Dest::p4 now doesn't try to `p4 submit` when no changes are pending. - VCP::Rev now prevents the same label from being applied twice to a revision. This was occuring because the "r_1"-style label that gets added to a target revision by VCP::Dest::p4 could duplicate a label "r_1" that happened to already be on a revision. - Added t/00rev.t, the beginnings of a test suite for VCP::Rev. - Tweaked bin/gentrevml to comment revisions with their change number instead of using a unique comment for every revision for non-p4 t/test-*-in-0.revml files. This was necessary to test cvs->p4 functionality. |
||
#1 | 467 | Barrie Slaymaker | Version 0.01, initial checkin in perforce public depot. |