#!/usr/local/bin/perl -w

=head1 NAME

vcp - Copy versions of files between repositories and/or RevML

=head1 SYNOPSIS

   vcp [vcp_opts] <source> <dest>

   vcp help [topic]

   vcp html <destination dir>

=head1 DESCRIPTION

C<vcp> ('version copy') copies versions of files from one repository to another,
translating as much metadata as possible along the way.  This allows you to
copy and translate files and their histories between revision storage systems.

Supported source and destination types are C<cvs:>, C<p4:>, and C<revml:>.

=head2 Copying Versions

The general syntax of the vcp command line is:

   vcp [<vcp options>] <source> <dest>

The three portions of the command line are:

=over

=item C<E<lt>vcp optionsE<gt>>

Command line options that control the operation of the C<vcp> command, like
C<-d> for debugging or C<-h> for help.  There are very few global options,
these are covered below. Note that they must come before the
C<E<lt>sourceE<gt>> specification.

=item C<E<lt>sourceE<gt>>

Were to extract versions from, including any command line options needed to
control what is extracted and how.  See the next section.

=item C<E<lt>destE<gt>>

Where to insert versions, including any command line options needed to control
how files are stored.  See the next section.

=back

=head2 Specifying Repositories

The C<E<lt>sourceE<gt>> and C<E<lt>destE<gt>> specifications specify a
repository and provide any options needed for accessing that repository.

These spefications may be a simple filename for reading or writing RevML
files (if the requisite XML handling modules are installed). or a full
repository specification like C<cvs:/home/cvs/root:module> or
C<p4:user:password@server:port://depot/dir>.

When using the long form to access a repository, C<E<lt>sourceE<gt>> and
C<E<lt>destE<gt>> specification have several fields delimited by C<:>
and C<@>, and may have trailing command line options.  The full (rarely
used) syntax is:

   scheme:user(view):password@repository:filespec [<options>]

where

=over

=item C<scheme:>

The repository type (C<p4:>, C<cvs:>, C<revml:>).

=item C<user>, C<view>, and C<password>

Optional values for authenticating with the repository and identifying which
view to use.  C<cvs> does not use C<view>.  For C<p4>, C<view> is the client
setting (equibalent to setting C<P4CLIENT> or using C<p4>'s C<-c> option).

=item C<repository>

The repository spec, CVSROOT for CVS or P4PORT for p4.

=item C<filespec>

Which versions of what files to move.  As much as possible, this spec is
similar to the native filespecs used by the repository indicated by the scheme.

=item C<E<lt>optionsE<gt>>

Command line options that usually mimic the options provided by the underlying
repositories' command line tools (C<cvs>, C<p4>, etc).

=back

Most of these fields are omitted in practice, only the C<scheme> field is
required, though (in most cases) the C<repository> field is also needed unless
you set the appropriate environment variables (C<CVSROOT>, C<P4PORT>, etc).

The a bit confusing, here are some examples specs:

   cvs:server:/foo
   p4:user@server://depot/foo/...
   p4:user:password@public.perforce.com:1666://depot/foo/...

Options and formats for of individual schemes can be found in the relevant
help topics, for instance:

   vcp help source::cvs

See C<vcp help> for a list of source and destination topics.

When reading and writing RevML files, a simple filename will do
(although the long form may also be used).  The special value "-" means
to read/write stdin and stdout when used as a source or destination
name, respectively.  "-" is assumed if a specification is not provided,
so these invocations all accomplish the same thing, reading and writing
RevML:

   vcp
   vcp -
   vcp revml:-
   vcp revml:
   vcp - -
   vcp - revml:-
   vcp - revml:
   vcp revml:- revml:-
   vcp revml: revml:

=head2 C<vcp> Options

All general options to vcp must precede the C<E<lt>sourceE<gt>>.
Scheme-specific options must be placed immediately after the
C<E<lt>sourceE<gt>> or C<E<lt>destE<gt>> spec and before the next one.

=over

=item --debug <spec>, -d <spec>

Enables display of debugging information.  A debug spec is part or all of a
module name like C<Source::revml> or a perl5 regular expression to match module
names.  Debug specs are not case insensitively.

The most general, show-me-everything debug option is:

   -d ".*"

The quotations are needed to slip the ".*" past most command shells.

Any debug specs that don't match anything during a run are printed out
when vcp exits in order to help identify mispelled patterns.  vcp will also list
all of the internal names that didn't match during a run to
give clues as to what specs might be useful.

The special name 'what' is guaranteed to not match anything, so you can
do

   vcp -d what ...

to see the list of names that might be useful for the arguments '...' .

You may use multiple
C<-d> options or provide a comma separated list to enable debugging
within that module. Do not start a pattern with a "-".

Debugging messages are emitted to stderr. See L</VCPDEBUG> for how to specify
debug options in the environment.

=item --help, -h, -?

These are all equivalent to C<vcp help>.

=back

=head2 Getting help

(See also L<Generating HTML Documentation|/Generating HTML Documentation>,
below).

There is a slightly different command line format for requesting help:

   vcp help [<topic>]

where C<E<lt>topicE<gt>> is the optional name of a topic.  C<vcp help> without
a C<E<lt>>topicC<E<gt>> prints out a list of topics, and C<vcp help vcp>
emits this page.

All help documents are also available as Unix C<man> pages and using the
C<perldoc> command, although the names are slightly different:

   with vcp               via perldoc        
   ================       ===========
   vcp help vcp           perldoc vcp
   vcp help source::cvs   perldoc VCP::Source::cvs
   vcp help source::cvs   perldoc VCP::Dest::p4

C<vcp help> is case insensitive, C<perldoc> and C<man> may or may not be
depending on your filesystem.  The C<man> commands look just like the example
C<perldoc> commands except for the command name.  Both have the advantage that
they use your system's configured pager if possible.

=head2 Environment Variables

=over

=item VCPDEBUG

The environment is often used to set context for the source and destination
by way of variables like P4USER, P4CLIENT, CVSROOT, etc.

There is also one environment variable that is used to enable
command line debugging.  The VCPDEBUG variable acts just like a leading
C<-d=$VCPDEBUG> was present on the command line.

   VCPDEBUG=main,p4

(see L<"--debug E<lt>specE<gt>, -d E<lt>specE<gt>"> for more info).  This is useful when VCP is
embedded in another application, like a makefile or a test suite.

=back

=head2 Generating HTML Documentation

All of the help pages in C<vcp> can be built in to an HTML tree with the
command:

   vcp html <dest_dir>

The index file will be C<E<lt>dest_dirE<gt>/index.html>.

=for test_scripts t/10vcp.t

=cut

use strict ;

use Getopt::Long ;
use File::Basename ;
use File::Spec ;
use VCP ;
use VCP::Debug qw( :debug ) ;
use VCP::Utils qw( shell_quote );
use XML::Doctype ;


{
   my $pname = basename( $0 ) ;
   my $dtd_spec ;
   my $arg = 'help' ;

#   usage_and_exit() unless @ARGV ;

   enable_debug( split /,/, $ENV{VCPDEBUG} ) if defined $ENV{VCPDEBUG} ;

   debug "vcp: ", join " ", map "'$_'", $pname, @ARGV if debugging "main" ;

   ## Parse up to the first non-option, then let sources & dests parse
   ## from there.
   Getopt::Long::Configure( qw( no_auto_abbrev no_bundling no_permute ) ) ;
   GetOptions(
      'debug|d=s'   => sub {
         enable_debug( length $_[1] ? split /,/, $_[1] : () )
      },
      'help|h|?'    => \&help_and_exit,
      'versions'    => \&versions_and_exit,
   ) or options_and_exit() ;

#   usage_and_exit() unless @ARGV ;

   $arg = shift;

   $arg = "-" unless defined $arg;

   build_html_tree_and_exit( $pname, @ARGV ) if $arg eq "html";

   help_and_exit( $pname, @ARGV ) if $arg eq 'help' ;

   my @errors ;

   ## We pass \@ARGV to the constructors for source and dest so that
   ## they may parse some of @ARGV and leave the rest.  Actually, that's
   ## only important for sources, since the dests should consume it all
   ## anyway.  But, for consistency's sake, I do the same to both.

   my $source ;
   my $vcp;
   my %vcp_spec;   ## The fields.
   my @vcp_spec;   ## The ordered fields, used to reconstitute the file
   my %branch_map; ## The parsed Branches field from the spec.
   my $vcp_spec_fn = "default.vcp";

   if (
      ( $arg =~ s/^vcp://
         || ( $arg ne "-"
            && ( $arg =~ /.vcp\z/i
               || (
                  $arg !~ /\.revml\z/i
                  && ( defined -S $arg ? -S _ : 1_000_000 ) < 1_000_000
               )
            )
         )
      )
      && do {
         warn "vcp: reading transfer specification from ",
            $arg eq "-" ? "stdin" : $arg,
            "\n"
         unless $arg =~ /\.vcp$/;
         if ( $arg eq "-" ) {
            ## Note: this can only occur if vcp:- was specified, not
            ## if "-" was specified (see the $arg ne "-" above).
            *VCPSPECFILE = \*STDIN;
         }
         else {
            open VCPSPECFILE, "<$arg" or die "$!: $arg\n";
         }
         die "$!: $arg\n"
            unless defined read VCPSPECFILE, $vcp, 1_000_000;
         close VCPSPECFILE;
         $vcp !~ m{<revml[^>]*>.*</revml>}m;
      }
   ) {
      $vcp_spec_fn = $arg unless $arg eq "-";

      require VCP::Utils::p4;
      %vcp_spec 
         = @vcp_spec
         = VCP::Utils::p4->parse_p4_form( $vcp );
      undef $vcp;
      goto EXTRAS if @ARGV;

      require Text::ParseWords;

      push @errors, "$arg does not specify a Source\n"
         unless length( $vcp_spec{Source} || "" );
      push @errors, "$arg does not specify a Destination\n"
         unless length( $vcp_spec{Destination} || "" );

      %branch_map = map {
         my @words = Text::ParseWords::shellwords( $_ );
         push @errors,
            "More than 3 fields in $arg Branch entry: " . shell_quote( @words )
            if @words > 3;
         push @errors,
            "Less than 3 fields in $arg Branch entry: " . shell_quote( @words )
            if @words < 3;
         ( $words[0] => [ $words[1], $words[2] ] )
      }  map { s/\s+\z//; s/\A\s+//; $_ } # strip leading/trailing ws
         grep /\S/,                       # Ignore blank lines
         map { s/(\A|\s+)#.*//; $_ }      # strip comments
         split /(\r?)+\n/, $vcp_spec{Branches}
         if exists $vcp_spec{Branches};

      goto ERRORS if @errors;

      @ARGV = (
         Text::ParseWords::shellwords( $vcp_spec{Source}      ),
         Text::ParseWords::shellwords( $vcp_spec{Destination} ),
      );
      $arg = shift;

      warn "vcp: ", shell_quote( $arg, @ARGV ), "\n";
   }

   if ( $arg =~ /^\w+:/ ) {
      my ( $scheme, $spec ) = $arg =~ /^([^:]+):(.*)/ ;

      ## This next one consumes all options up to the dest scheme.
      $source = load_module( "VCP::Source::$scheme", $arg, \@ARGV );
      die "unknown source scheme '$scheme:', try ",
         list_modules( "VCP::Source" ), "\n"
         unless defined $source ;
   }
   else {
      $arg ||= "-";
      $source = load_module( "VCP::Source::revml", "revml:$arg", \@ARGV ), 
   }

   my $dest ;
   my $dest_needed = defined $source && $source->dest_expected;

   if ( $dest_needed ) {
      $arg = @ARGV ? shift : "-" if $dest_needed;

      if ( defined $arg && $arg =~ /^\w+:/ ) {
         my ( $scheme, $spec ) = $arg =~ /^([^:]+):(.*)/ ;

         ## This next one consumes all options up to the dest scheme.
         $dest = load_module( "VCP::Dest::$scheme", $arg, \@ARGV );
         die "unknown destination scheme '$scheme:', try ",
            list_modules( "VCP::Dest" ), "\n"
            unless defined $dest ;
      }
      else {
         $dest = load_module( "VCP::Dest::revml", "revml:$arg", \@ARGV ), 
      }
   }

   if ( @ARGV ) {
EXTRAS:
      push @errors, "extra parameters: " . join( ' ', @ARGV ) . "\n" ;
   }

   if ( debugging ) {
      debug 'vcp: no dest expected' unless ! $source || $source->dest_expected ;
      debug 'vcp: $source is ', $source ;
      debug 'vcp: $dest   is ',   $dest ;
   }

   unless ( @errors ) {
      my $cp = VCP->new( $source, $dest ) ;
      my $header = {} ;
      my $footer = {} ;
      $cp->copy_all( $header, $footer ) ;
   }

ERRORS:

   if ( @errors ) {
      my $errors = join( '', @errors ) ;
      $errors =~ s/^/$pname: /mg ;
      die $errors ;
   }

}

###############################################################################
###############################################################################

sub load_module {
   my ( $name, @args ) = @_ ;

   my $filename = $name ;
   $filename =~ s{::}{/}g ;

   my $x ;
   {
      local $@ ;
      my $v = eval "require '$filename.pm'; 1" ;
      return undef if $@ && $@ =~ /^Can't locate $filename.pm/ ;
      $x = $@ ;
   }
   die $x if $x ;

   debug "vcp: loaded '$name' from '", $INC{"$filename.pm"}, "'"
      if debugging 'main', $name ;
   return $name->new( @args ) ;#if $v == 1 ;
}


sub list_modules {
   my ( $prefix ) = @_ ;

   my $dirname = $prefix . '::' ;
   $dirname =~ s{(::)+}{/}g ;

   my %seen ;
   for ( @INC ) {
      my $dir = File::Spec->catdir( $_, $dirname ) ;
      opendir( D, $dir ) or next ;
      my @files = grep $_ !~ /^\.\.?$/ && s/\.pm$//i, readdir D ;
      closedir D ;
      $seen{$_} = 1 for @files ;
   }

   my $list = join( ', ', map "$_:", sort keys %seen ) ;

   $list =~ s/,([^,]*)$/ or$1/ ;
   return $list ;
}


sub usage_and_exit {
   require Pod::Usage ;
   Pod::Usage::pod2usage( -message => shift, -verbose => 0, -exitval => 1 ) ;
}

sub options_and_exit {
   require Pod::Usage ;
   Pod::Usage::pod2usage( -verbose => 1, -exitval => 1 ) ;
}


sub find_help_modules {
   my ( $desired_module ) = @_;

   require File::Find;

   my %modules;

   for my $inc_dir ( @INC ) {
      $inc_dir = File::Spec->rel2abs( $inc_dir );

      my $vcp_file = File::Spec->catfile( $inc_dir, "VCP.pm" );
      $modules{VCP} ||= $vcp_file if -f $vcp_file;

      my $vcp_dir  = File::Spec->catdir(  $inc_dir, "VCP"    );
      next unless -d $vcp_dir;

      File::Find::find(
         sub {
            return if -d $_;
            return unless /\.(pm|pod)\Z/i;

            my $mod_name = File::Spec->abs2rel( $File::Find::name, $vcp_dir );

            $mod_name =~ s{[:\\/]+}{::}g;
            $mod_name =~ s{\.(pm|pod)}{}i;

            if ( defined $desired_module && lc $mod_name eq $desired_module ) {
               die "FOUND $File::Find::name\n";
            }
            else {
               $modules{$mod_name} ||= $File::Find::name;
            }

         },
         $vcp_dir
      )
   }

   return %modules;
}


sub help_and_exit {
   require Pod::Usage ;

   my ( $prog_name, $topic ) = @_;

   my $result = 0;

   if ( defined $topic ) {
      $topic = lc $topic;

      if ( $topic eq "vcp" ) {
          system( "pod2text", $0 );
          exit $result;
      }

      eval {
         find_help_modules( $topic );
      };
      if ( $@ =~ /FOUND (.*)/ ) {
         exit system( "pod2text", $1 ) >> 8;
      }
      elsif ( $@ ) {
          die $@;
      }

      $result = 1;
      warn "Unrecognized help topic '$topic'\n";
   }

    print <<END_HELP_TOPICS;

$prog_name - Version Copy, a tool for copying versions file repositories

help topics (use "vcp help <topic>" to see):

   vcp            General help for the vcp command

   source::p4     Extracting from a p4 repository
   dest::p4       Inserting in to a p4 repository
   source::cvs    Extracting from a cvs repository
   dest::cvs      Inserting in to a cvs repository

   newlines       Newline, ^Z and NULL issues
   process        How $prog_name works

   license        Copyright and license information
   maintenance    VCP Code maintenance, debugging tips & tricks

END_HELP_TOPICS
   exit $result;
}


sub build_html_tree_and_exit {
   my ( $prog_name, $dest_dir ) = @_;

   unless ( defined $dest_dir && length $dest_dir ) {
       $dest_dir = $prog_name . "_html";
   }

   $dest_dir = File::Spec->rel2abs( $dest_dir );

   $| = 1;
   print "Generating HTML in $dest_dir/";

   my %modules = find_help_modules;

   require Pod::Links;
   require Pod::HTML_Elements;
   require File::Path;
   require IO::File;

   ## BEGIN CODE ADAPTED FROM NICK ING-SIMMONS' PodToHTML package
   my $links = Pod::Links->new();
   for my $fn (
      $0,
      grep /Source[^.]|Dest[^.]|\.pod/, values %modules
   ) {
      print ".";
      $links->parse_from_file($fn);
   }

   for my $name ($links->names) {
      $links->link(
         $name,
         do {
            my $outfile = $name;
            $outfile =~ s#::#/#g;
            $outfile =~ s#[^/a-z0-9A-Z._-]#_#g;
            $outfile .= ".html";
            File::Spec->catfile( $dest_dir, $outfile );
         }
      ) if $links->pod($name);
   }

   my $index_file = File::Spec->catfile( $dest_dir, "index.html" );

   my $parser = Pod::HTML_Elements->new(
      Index => $index_file,
      Links => $links,
   );

   ## the sort {} makes sure "vcp" is listed first in the
   ## resulting index.
   for my $name (
      sort {
         $a eq "vcp"
            ? -1
            : $b eq "vcp"
               ? 1
               : $a cmp $b
      } $links->names
   ) {
      print ".";
      my $file = $links->pod($name);
      my $outfile = $links->link($name);
      if (defined $file) {
         File::Path::mkpath( File::Basename::dirname( $outfile ), 0, 0755 );
         $parser->parse_from_file($file,$outfile);
      }
   }
   $parser->write_index;
   ## END CODE ADAPTED FROM NICK ING-SIMMONS' PodToHTML package
   print "\n";

   print "Finished, index file is $index_file\n";

   exit( 0 );
}


sub versions_and_exit {
   require File::Find ;

   my $require_module = sub {
      return unless m/\.pm$/i ;
      ## Avoid "name used only once" warning
      my $fn = $File::Find::name ;
      $fn = $File::Find::name ;
      require $fn ;
   } ;

   File::Find::find(
      {
         no_chdir => 1,
	 wanted   => $require_module,
      },
      grep -d $_,
      map {
         ( File::Spec->catdir( $_, "lib", "VCP", "Source" ),
         File::Spec->catdir( $_, "lib", "VCP", "Dest" ),
	 ) ;
      } @INC
   ) ;

   my %vers ;
   my %no_vers ;

   my $recur ;
   $recur = sub {
      my ( $pkg_namespace ) = @_ ;

      no strict "refs" ;

      my $pkg_name = substr( $pkg_namespace, 0, -2 ) ;

      ## The grep means "only bother with namespaces that contain somthing
      ## other than child namespaces.
      if ( ! grep /::/, keys %{$pkg_namespace} ) {
         if ( exists ${$pkg_namespace}{VERSION} ) {
	    $vers{$pkg_name} = ${"${pkg_namespace}VERSION"}
	 }
	 else {
	    $no_vers{$pkg_name} = undef ;
	 }
      }

      my $prefix = $pkg_namespace eq "main::" ? "" : $pkg_namespace ;
      for ( keys %{$pkg_namespace} ) {
	 next unless /::$/ ;
	 next if /^main::/ ;
	 $recur->( "$prefix$_" ) ;
      }
   } ;

   $recur->( "main::" ) ;

   my $max_len = 0 ;
   $max_len = length > $max_len ? length : $max_len for keys %vers ;
      
   print "Package \$VERSIONs:\n" ;
   for ( sort keys %vers ) {
      printf(
         "   %-${max_len}s: %s\n",
	 $_,
	 defined $vers{$_} ? $vers{$_} : "undef"
      ) ;
   }

   print "No \$VERSION found for: ", join( ", ", sort keys %no_vers ), "\n" ;

   $max_len = 0 ;
   $max_len = length > $max_len ? length : $max_len for values %INC ;
   print "\nFile sizes:\n" ;
   for ( sort values %INC ) {
      printf( "   %-${max_len}s: %7d\n", $_, -s $_ ) ;
   }

   print "\nperl -V:\n" ;

   my $v = `$^X -V` ;
   $v =~ s/^/   /gm ;
   print $v ;

   exit ;
}

=head1 SEE ALSO

L<VCP::Process>, L<VCP::Newlines>, L<VCP::Source::p4>, L<VCP::Dest::p4>,
L<VCP::Source::cvs>, L<VCP::Dest::cvs>, L<VCP::Source::revml>,
L<VCP::Dest::revml>, L<VCP::Newlines>.  All are also available using C<vcp
help>.

=head1 AUTHOR

Barrie Slaymaker <barries@slaysys.com>

=head1 COPYRIGHT

Copyright (c) 2000, 2001, 2002 Perforce Software, Inc.
All rights reserved.

See L<VCP::License|VCP::License> (C<vcp help license>) for the terms of use.

=cut