#!/usr/local/bin/perl -w =head1 NAME vcp - Copy versions of files between repositories and/or RevML =head1 SYNOPSIS vcp [vcp_opts] src[:files] [src_opts] [dest:[<location>]] [dest_opts] vcp cvs:/mymodule/... -r 1.1: p4://depot/mymodule vcp p4://depot/mainbranch/...@100-105 revml: vcp help vcp revml: --dtd <foo.dtd> --save_dtd =head1 STATUS Alpha code. Please report bugs to revml@perforce.com. =head1 DESCRIPTION vcp ('version copy') copies versions and version ranges of files from one repository to another, translating as much as possible along the way. This allows you to copy and translate ranges of revisions of files between different vendors' revision storage systems. Inputs and output classes currently available are: Class Source Destination cvs x x p4 x x revml x x The general syntax of the vcp command line is: vcp [vcp_opts] <source> [src_opts] <dest> [dest_opts] where C<E<lt>sourceE<gt>> and C<E<lt>destE<gt>> are composed of fields separated by delimiters like so: scheme:user(view):password@server:filespec where scheme is a repository type ('p4', 'cvs', 'revml'), or special command ('help', 'save_dtd'). user, view, and password are optional values, one or more of which may be required for repository access. CVS does not use (view). For p4, (view) is the client setting (P4CLIENT or -c option). server is the repository spec, CVSROOT for CVS or P4PORT for p4. filespec is the file specification for the files to move. As much as possible, this spec is similar to the native filespecs used by the repository indicated by the scheme. Most C<E<lt>sourceE<gt>> and C<E<lt>destE<gt>> specs will omit one or more of the fields. For instance, Filespecs or passwords are often omitted from the destination, in which case the leading colons for these fields are also optional. If no user, view, or password is supplied, the "@" is optional. That's a bit confusing, here are some examples of stripped-down specs: cvs:server:/foo p4:user@server://depot/foo/... p4:user:password@public.perforce.com:1666://depot/foo/... Options and formats for of individual schemes can be found in the relevant manpages: L<VCP::Source::cvs>, L<VCP::Source::p4>, L<VCP::Source::revml>, L<VCP::Dest::cvs>, L<VCP::Dest::p4>, L<VCP::Dest::revml>. At some point, the help command will be extended to be able to display them. =head2 OPTIONS All general options to vcp must precede the first scheme. Scheme-specific options must come after the affected source or destination spec and before the next one. =over =item --debug <spec>, -d <spec> Enables display of debugging information. A debug spec is part or all of a module name like C<Source::revml> or a perl5 regular expression to match module names. Debug specs are not case insensitively. The most general, show-me-everything debug option is: -d ".*" The quotations are needed to slip the ".*" past most command shells. Any debug specs that don't match anything during a run are printed out when vcp exits in order to help identify mispelled patterns. vcp will also list all of the internal names that didn't match during a run to give clues as to what specs might be useful. The special name 'what' is guaranteed to not match anything, so you can do vcp -d what ... to see the list of names that might be useful for the arguments '...' . You may use multiple C<-d> options or provide a comma separated list to enable debugging within that module. Do not start a pattern with a "-". Debugging messages are emitted to stderr. See L</VCPDEBUG> for how to specify debug options in the environment. =item --help, -h, -? These are all equivalent to C<vcp help>. =back =head1 ARGUMENTS There are two special schemes, "help" and "save_dtd". =over =item help Displays the full help text. This will be extended to allow the retrieval of all of the manpages provided with VCP. =item save_dtd [<how>] This is primarily for vcp maintainers. Outputs the DTD to stdout if <how> is C<->, or to a file named like v1_000.pm if <how> looks like a version number, or in a module named after <how> if <how> contains '::'. This file is placed in ./lib/RevML/DTD or ./RevML/DTD or ./, whichever is found first. No directories will be created. =back =head1 ENVIRONMENT The environment is often used to set context for the source and destination by way of variables like P4USER, P4CLIENT, CVSROOT, etc. There is also one environment variable that is used to enable command line debugging. The VCPDEBUG variable acts just like a leading "-d=$VCPDEBUG" was present on the command line. VCPDEBUG=main,p4 (see L</--debug, -d> for more info). This is useful when VCP is embedded in another application, like a makefile or a test suite. =cut use strict ; use Getopt::Long ; use File::Basename ; use File::Spec ; use VCP ; use VCP::Debug qw( :debug ) ; use XML::Doctype ; { my $pname = basename( $0 ) ; my $dtd_spec ; my $arg = 'help' ; usage_and_exit() unless @ARGV ; enable_debug( split /,/, $ENV{VCPDEBUG} ) if defined $ENV{VCPDEBUG} ; ## Parse up to the first non-option, then let sources & dests parse ## from there. Getopt::Long::Configure( qw( no_auto_abbrev no_bundling no_permute ) ) ; GetOptions( 'debug|d=s' => sub { enable_debug( length $_[1] ? split /,/, $_[1] : () ) }, 'help|h|?' => \&help_and_exit, 'versions' => \&versions_and_exit, ) or options_and_exit() ; usage_and_exit() unless @ARGV ; $arg = shift ; help_and_exit() if $arg eq 'help' ; my @errors ; ## We pass \@ARGV to the constructors for source and dest so that ## they may parse some of @ARGV and leave the rest. Actually, that's ## only important for sources, since the dests should consume it all ## anyway. But, for consistency's sake, I do the same to both. my $source ; if ( defined $arg ) { my ( $scheme, $spec ) = $arg =~ /^(.*?)(?::(.*))?$/ ; if ( defined $spec && ! length $spec && @ARGV && ( $ARGV[0] eq '-' || substr( $ARGV[0], 0, 1 ) ne '-' ) && index( $ARGV[0], ':' ) < 0 ) { $spec = shift ; } $spec = defined $spec ? "$scheme:$spec" : $scheme ; eval { $source = load_module( "VCP::Source::$scheme", $spec, \@ARGV ); die "unknown source scheme '$scheme', try ", list_modules( "VCP::Source" ), "\n" unless defined $source ; } ; push @errors, $@ if $@ ; } my $dest ; if ( defined $source ? $source->dest_expected : @ARGV ) { my $scheme ; my $spec = '' ; if ( @ARGV ) { ( $scheme, $spec ) = shift =~ /^(.*?)(?::(.*))?$/ ; if ( defined $spec && ! length $spec && @ARGV && ( $ARGV[0] eq '-' || substr( $ARGV[0], 0, 1 ) ne '-' ) && index( $ARGV[0], ':' ) < 0 ) { $spec = shift ; } } else { $scheme = 'revml' ; } $spec = defined $spec ? "$scheme:$spec" : $scheme ; eval { $dest = load_module("VCP::Dest::$scheme", $spec, \@ARGV ); die "unknown destination scheme '$scheme', try ", list_modules( "VCP::Dest" ), "\n" unless defined $dest ; } ; push @errors, $@ if $@ ; @ARGV = () ; } elsif ( @ARGV ) { push @errors, "extra parameters: " . join( ' ', @ARGV ) . "\n" ; } if ( debugging ) { debug 'vcp: no dest expected' unless ! $source || $source->dest_expected ; debug 'vcp: $source is ', $source ; debug 'vcp: $dest is ', $dest ; } unless ( @errors ) { my $cp = VCP->new( $source, $dest ) ; my $header = {} ; my $footer = {} ; $cp->copy_all( $header, $footer ) ; } if ( @errors ) { my $errors = join( '', @errors ) ; $errors =~ s/^/$pname: /mg ; die $errors ; } } ############################################################################### ############################################################################### sub load_module { my ( $name, @args ) = @_ ; my $filename = $name ; $filename =~ s{::}{/}g ; my $x ; { local $@ ; my $v = eval "require '$filename.pm'; 1" ; return undef if $@ && $@ =~ /^Can't locate $filename.pm/ ; $x = $@ ; } die $x if $x ; debug "vcp: loaded '$name' from '", $INC{"$filename.pm"}, "'" if debugging 'main', $name ; return $name->new( @args ) ;#if $v == 1 ; } sub list_modules { my ( $prefix ) = @_ ; my $dirname = $prefix . '::' ; $dirname =~ s{(::)+}{/}g ; my %seen ; for ( @INC ) { my $dir = File::Spec->catdir( $_, $dirname ) ; opendir( D, $dir ) or next ; my @files = grep $_ !~ /^\.\.?$/ && s/\.pm$//, readdir D ; closedir D ; $seen{$_} = 1 for @files ; } my $list = join( ', ', map "$_:", sort keys %seen ) ; $list =~ s/,([^,]*)$/ or$1/ ; return $list ; } sub usage_and_exit { require Pod::Usage ; Pod::Usage::pod2usage( -message => shift, -verbose => 0, -exitval => 1 ) ; } sub options_and_exit { require Pod::Usage ; Pod::Usage::pod2usage( -verbose => 1, -exitval => 1 ) ; } sub help_and_exit { require Pod::Usage ; Pod::Usage::pod2usage( -verbose => 2, -exitval => 0 ) ; } sub versions_and_exit { require File::Find ; my $require_module = sub { return unless m/\.pm$/ ; ## Avoid "name used only once" warning my $fn = $File::Find::name ; $fn = $File::Find::name ; require $fn ; } ; File::Find::find( { no_chdir => 1, wanted => $require_module, }, grep -d $_, map { ( File::Spec->catdir( $_, "lib", "VCP", "Source" ), File::Spec->catdir( $_, "lib", "VCP", "Dest" ), ) ; } @INC ) ; my %vers ; my %no_vers ; my $recur ; $recur = sub { my ( $pkg_namespace ) = @_ ; no strict "refs" ; my $pkg_name = substr( $pkg_namespace, 0, -2 ) ; ## The grep means "only bother with namespaces that contain somthing ## other than child namespaces. if ( ! grep /::/, keys %{$pkg_namespace} ) { if ( exists ${$pkg_namespace}{VERSION} ) { $vers{$pkg_name} = ${"${pkg_namespace}VERSION"} } else { $no_vers{$pkg_name} = undef ; } } my $prefix = $pkg_namespace eq "main::" ? "" : $pkg_namespace ; for ( keys %{$pkg_namespace} ) { next unless /::$/ ; next if /^main::/ ; $recur->( "$prefix$_" ) ; } } ; $recur->( "main::" ) ; my $max_len = 0 ; $max_len = length > $max_len ? length : $max_len for keys %vers ; print "Package \$VERSIONs:\n" ; for ( sort keys %vers ) { printf( " %-${max_len}s: %s\n", $_, defined $vers{$_} ? $vers{$_} : "undef" ) ; } print "No \$VERSION found for: ", join( ", ", sort keys %no_vers ), "\n" ; $max_len = 0 ; $max_len = length > $max_len ? length : $max_len for values %INC ; print "\nFile sizes:\n" ; for ( sort values %INC ) { printf( " %-${max_len}s: %7d\n", $_, -s $_ ) ; } print "\nperl -V:\n" ; my $v = `$^X -V` ; $v =~ s/^/ /gm ; print $v ; exit ; } =head1 AUTHOR Barrie Slaymaker <barries@slaysys.com> =head1 COPYRIGHT This program is licensed under the provisions of the BSD license: Copyright (c) 2000, 2001, Perforce Software, Inc. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: - Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. - Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. - Neither the name of the Perforce Software, Inc.nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. This notice is also included in the LICENSE file accompanying this distribution, which also must be retained in redistributions. =cut
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#80 | 4969 | Barrie Slaymaker |
- Added a special command line macro "default_filters:" added to allow easy tests of, for instance, cvs to p4. |
||
#79 | 4967 | Barrie Slaymaker |
- Removed "-d", leaving only "--debug" so that "-d" is properly passed through to plugins (esp. VCP::Source::cvs). |
||
#78 | 4581 | Barrie Slaymaker | - user prompts have been improved, but not tested | ||
#77 | 4498 | Barrie Slaymaker | Work around odd bug on RH8, perl5.8.3 | ||
#76 | 4493 | Barrie Slaymaker | - Attempted to work around perl5.8.4 Bizarre copy of ARRAY bug | ||
#75 | 4482 | Barrie Slaymaker | - spurious banner quieted | ||
#74 | 4404 | Barrie Slaymaker | - Added -q option (mostly to suppress banner & progress bars for test suite) | ||
#73 | 4236 | Barrie Slaymaker | - banner not emitted when writing revml to STDOUT | ||
#72 | 4232 | Barrie Slaymaker | - bin/vcp now prints a version info banner | ||
#71 | 4154 | Barrie Slaymaker | - dist/vcp.exe passes almost all tests | ||
#70 | 4150 | Barrie Slaymaker | - bin/vcp can now be tested away from its lib/ dir | ||
#69 | 4148 | Barrie Slaymaker | - scan_modules() returns a list, dummy | ||
#68 | 4143 | Barrie Slaymaker | - Further adaptation to vcp.exe packaging format | ||
#67 | 4141 | Barrie Slaymaker | - Adapt online help and html generation to vcp.exe environment | ||
#66 | 4065 | Barrie Slaymaker |
- bin/vcp no longer whinges about undefined values when running the interactive UI. |
||
#65 | 3855 | Barrie Slaymaker |
- vcp scan, filter, transfer basically functional - Need more work in re: storage format, etc, but functional |
||
#64 | 3779 | Barrie Slaymaker |
- bin/vcp has nascent support for 3 stage transfers: - scan the source - filter the metadata (modify or prune) - transfer the filtered results - PRELIMINARY |
||
#63 | 3725 | Barrie Slaymaker | - Minor spelling correction | ||
#62 | 3697 | Barrie Slaymaker | - Write out new source if editing .vcp file changed source | ||
#61 | 3676 | Barrie Slaymaker |
- The user interface should no longer append a second Dest: section. |
||
#60 | 3672 | Barrie Slaymaker | - vcp emits more meaningful errors when fed a 0 length .vcp file | ||
#59 | 3666 | Barrie Slaymaker | - vcp can now edit existing .vcp files, for VSS sources and revml dests | ||
#58 | 3663 | Barrie Slaymaker |
- Yes/no questions may be answered 'y' or 'n' (case insensitive) - Multiple choice questions may be answered by typing in full text of an answer (case insensitive) |
||
#57 | 3649 | Barrie Slaymaker | - Stray DB::single = 1s removed | ||
#56 | 3644 | Barrie Slaymaker | - Add Save & Run options to end of UI | ||
#55 | 3640 | Barrie Slaymaker |
- xmllint no longer require to build UI - UI now offers multiple choices where appropriate |
||
#54 | 3636 | Barrie Slaymaker | - --terse option description improved | ||
#53 | 3567 | John Fetkovich |
- added the field UIManager in VCP::UI::Text.pm - added the fields UIImplementation and TersePrompts in UI.pm - removed Source and Dest fields in VCP::UI.pm - UI.pm now returns the result of running the UI implementation. - VCP::UI::Text->run return a list of (source, dest) all future UI implementations must do the same. - bin/vcp gets (source, dest) list from VCP::UI->run. - added --terse (or -t) command line option to vcp to remove verbose help from interactive UI. |
||
#52 | 3548 | John Fetkovich | Load default filters when vcp interactive ui is run. | ||
#51 | 3495 | John Fetkovich | improved help message at start of interactive vcp | ||
#50 | 3488 | John Fetkovich | move init of each plugin after writing config file | ||
#49 | 3487 | Barrie Slaymaker | - Add --dont-convert option to bin/vcp | ||
#48 | 3486 | John Fetkovich | moved (source or dest)->init calls to bin/vcp | ||
#47 | 3481 | John Fetkovich |
intro text moved out of state machine to bin/vcp. no longer requires user interaction to move on. |
||
#46 | 3479 | John Fetkovich | doc fix | ||
#45 | 3468 | Barrie Slaymaker |
- get --output-config-file working with ui (if ui completes, which may not happen if the user enters invalid input). - Note various UI issue in TODO. |
||
#44 | 3466 | Barrie Slaymaker |
- --output-config-file now usable with vcp:- specification - --output-config-file now tested - VCP::{Source,Dest}::null now sets a repo_scheme so they may be emitted to config files. - Dest: is now emitted right after Source: when there are filters to dump. |
||
#43 | 3464 | Barrie Slaymaker |
- Create VCP::ConfigFileUtils and move bin/vcp::parse_config_file in to it. - Add VCP::ConfigFileUtils::write_config_file() - Add --output-vcp-file to bin/vcp. - Add VCP::Driver::repo_spec_as_string() - Add VCP::Driver::config_file_section_as_string() - VCP::Driver::parse_repo_spec() now clears any settings that are not set by a given spec string (so old values don't remain after a call to it). |
||
#42 | 3441 | Barrie Slaymaker | - Add some debugging code | ||
#41 | 3406 | Barrie Slaymaker | - bin/vcp tries harder to keep backslashes for Win32 compat | ||
#40 | 3364 | John Fetkovich | clean up code flow in options parsing/interactive-ui calling section | ||
#39 | 3363 | John Fetkovich | cleanup code flow, remove some debugging statements | ||
#38 | 3362 | John Fetkovich | revml source and dest now works through interactive UI | ||
#37 | 3271 | John Fetkovich |
Added stub init() in VCP/Plugin.pm, and call to it in load_module in bin/vcp. This is in preparation for Sources and Dests to have parts of their current constructors split out into init() functions which will facilitate setting of fields from the interative ui code. |
||
#36 | 3244 | Barrie Slaymaker |
Integrate VCP::UI with bin/vcp. Type 'vcp' to run the UI. |
||
#35 | 3167 | Barrie Slaymaker |
Add profiling report that details various chunks of time taken. |
||
#34 | 3155 | Barrie Slaymaker |
Convert to logging using VCP::Logger to reduce stdout/err spew. Simplify & speed up debugging quite a bit. Provide more verbose information in logs. Print to STDERR progress reports to keep users from wondering what's going on. Breaks test; halfway through upgrading run3() to an inline function for speed and for VCP specific features. |
||
#33 | 3018 | Barrie Slaymaker |
fix command line parsing: options were not being cleansed from @ARGV properly. |
||
#32 | 2844 | John Fetkovich | Made VCP help aware of PAGER environment variable. | ||
#31 | 2711 | Barrie Slaymaker | remove debugging statement, fix spurios error message | ||
#30 | 2708 | Barrie Slaymaker | Add support for Options: in .vcp files, improve testing of CLI | ||
#29 | 2705 | Barrie Slaymaker | Clean up comments | ||
#28 | 2688 | Barrie Slaymaker |
Add back $ENV{VCPDEBUG} and augment with turning on debugging for IPC::Run3. |
||
#27 | 2639 | John Fetkovich |
Added profiling to be made active when VCPPROFILE environment variable turned on. writes profile info to filename defined in VCPPROFILE. Put some profiling statements (activated at compile time) in vcp and p4.pm. |
||
#26 | 2453 | John Fetkovich |
removed compilation of revml. will be making that a separate executable. |
||
#25 | 2354 | Barrie Slaymaker | favor .pod files over .pm files in the vcp help system. | ||
#24 | 2307 | Barrie Slaymaker | get VCP::Filter::map working, update docs | ||
#23 | 2306 | Barrie Slaymaker | tweak error text | ||
#22 | 2304 | Barrie Slaymaker | VCP::Filter::map docs and infrastructure | ||
#21 | 2302 | Barrie Slaymaker | test, debug filter chain CLI parsing | ||
#20 | 2301 | Barrie Slaymaker | A chain of plugins instead of source & dest | ||
#19 | 2297 | Barrie Slaymaker | Adapt cvs, revml to a more realistic branching structure | ||
#18 | 2293 | Barrie Slaymaker | Update CHANGES, TODO, improve .vcp files, add --init-cvs | ||
#17 | 2282 | Barrie Slaymaker | Add in support for *.vcp files | ||
#16 | 2014 | Barrie Slaymaker |
Give helpful error messages if the vcp command can't read/write RevML due to a missing required module. |
||
#15 | 2013 | Barrie Slaymaker | Reenable vcp as a unixy command line filter processing revml | ||
#14 | 1759 | Barrie Slaymaker | Don't try to keep parsing cmd line after unknown scheme | ||
#13 | 1506 | Barrie Slaymaker | tweaks | ||
#12 | 1367 | Barrie Slaymaker | lots of docco updates | ||
#11 | 1359 | Barrie Slaymaker | Revamp the help system, clean up and add POD | ||
#10 | 703 | Barrie Slaymaker | VCP::Source::p4 now uses VCP::Utils::p4::parse_p4_repo_spec() | ||
#9 | 692 | Barrie Slaymaker |
Add VCP::Utils::p4 and use it to get VCP::Dest::p4 to create it's own client view as needed. |
||
#8 | 688 | Barrie Slaymaker | Fixed docos for --debug. | ||
#7 | 628 | Barrie Slaymaker | Cleaned up POD in bin/vcp, added BSD-style license. | ||
#6 | 627 | Barrie Slaymaker | Beef up CVS log file parsing. | ||
#5 | 624 | Barrie Slaymaker | Add a space to bin/vcp SYNOPSIS after the cvs -r option. | ||
#4 | 613 | Barrie Slaymaker | Tweak README and documentation. | ||
#3 | 480 | Barrie Slaymaker |
0.06 Wed Dec 20 23:19:15 EST 2000 - bin/vcp: Added --versions, which loads all modules and checks them for a $VERSION and print the results out. This should help with diagnosing out-of-sync modules. - Added $VERSION vars to a few modules :-). Forgot to increment any $VERSION strings. - VCP::Dest::cvs: The directory "deeply" was not being `cvs add`ed on paths like "a/deeply/nested/file", assuming "deeply" had no files in it. - VCP::Dest::revml: fixed a bug that was causing files with a lot of linefeeds to be emitted in base64 instead of deltaed. This means most text files. - Various minor cleanups of diagnostics and error messages, including exposing "Can't locate Foo.pm" when a VCP::Source or VCP::Dest module depends on a module that's not installed, as reported by Jeff Anton. |
||
#2 | 468 | Barrie Slaymaker |
- VCP::Dest::p4 now does change number aggregation based on the comment field changing or whenever a new revision of a file with unsubmitted changes shows up on the input stream. Since revisions of files are normally sorted in time order, this should work in a number of cases. I'm sure we'll need to generalize it, perhaps with a time thresholding function. - t/90cvs.t now tests cvs->p4 replication. - VCP::Dest::p4 now doesn't try to `p4 submit` when no changes are pending. - VCP::Rev now prevents the same label from being applied twice to a revision. This was occuring because the "r_1"-style label that gets added to a target revision by VCP::Dest::p4 could duplicate a label "r_1" that happened to already be on a revision. - Added t/00rev.t, the beginnings of a test suite for VCP::Rev. - Tweaked bin/gentrevml to comment revisions with their change number instead of using a unique comment for every revision for non-p4 t/test-*-in-0.revml files. This was necessary to test cvs->p4 functionality. |
||
#1 | 467 | Barrie Slaymaker | Version 0.01, initial checkin in perforce public depot. |