p4table.rb #1

#--
#*******************************************************************************
#++
# = Introduction
#
# Classes implementing a generic database model on top of plain text files
# stored in Perforce. 
#
# Every record consists of fields/values and may have associated files. The
# data for each record is stored as both a job and a text file. The text
# file is always called  <id>/record and any associated files are also
# stored alongside the record file. i.e. <id>/file1 <id>/file2.
#
# Each table name is a folder name and all records for that table are stored
# in text files under that folder. i.e. <table>/<id>/record
#
# All column names must also be fields in the jobspec so you can search for 
# records using "p4 jobs -e". 
# 
# *NOTE*:: When creating your jobspec, make sure that *ALL* field names are 
#          in lowercase. Ruby likes method names to be lowercase and it fits 
#          better as well as reducing scope for errors.
#
# The job version of any record contains only the latest version of that
# record for searching purposes.
#
# *WARNING*:: This software is designed to have its OWN Perforce server to
#             work with. <b>DO NOT USE IT AGAINST AN EXISTING PERFORCE 
#             SERVER.</b>
#
# For a quick start, take a look at the P4Table class as all your tables
# should be derived from it and the Jobspec rules below.
#
# = Jobspec Rules
#
# When creating your jobspec observe the following rules for any chance of
# success.
#
# * All fieldnames *must* be lowercase
# * The jobspec must contain a field called "job"
# * The jobspec *must* contain a field called "files" of type "text"
# * Make most fields optional - especially if you have more than one table
# * Do not use invalid preset values in your jobspec. i.e. a field which
#   may only contain certain values but is deliberately initialised to an
#   invalid value to force users to change it. P4/Ruby can't handle
#   jobspecs like this at the moment.
#
# = Jobspec Example
#
# Below is the jobspec used for a FAX tracking application based on P4Ruby
# which should give you some idea of the possibilities.
#
#  # A Perforce Job Spec Specification.
#  #
#  # Updating this form can be dangerous!
#  # See 'p4 help jobspec' for proper directions.
#
#  Fields:
#         101 job word 32 required
#         102 status select 10 required
#         103 owner word 32 required
#         104 modified_date date 20 always
#         105 note text 0 optional
#         106 sender line 32 optional
#         107 fax_number line 32 optional
#         108 files text 80 optional
#         109 date_received date 20 once
#         110 date_closed date 20 optional
# 
#  Values:
#         status open/closed
# 
#  Presets:
#         status open
#         owner $user
#         modified_date $now
#         date_received $now
# 
#  Comments:
#
#--
#*******************************************************************************
#++
require "P4"


#-------------------------------------------------------------------------------
# START OF CONFIGURATION SECTION
#-------------------------------------------------------------------------------

#-------------------------------------------------------------------------------
#* Directory separator string for your platform
#-------------------------------------------------------------------------------
if ( RUBY_PLATFORM =~ /win32/i )
    SEP = "\\"
    NL = "\r\n"
else
    SEP = "/"
    NL  = "\n"
end

#-------------------------------------------------------------------------------
# Specifies in depot syntax the depot path that is to be treated as the
# top of the database. This path MUST be mapped to the client root.
#-------------------------------------------------------------------------------
DEPOT_ROOT_PATH	= "//depot"


#-------------------------------------------------------------------------------
#* END OF CONFIGURATION SECTION
#-------------------------------------------------------------------------------


#
# This class just provides two globally accessible Perforce client instances
# without the use of global variables. One P4 client runs in tagged mode
# and one in untagged mode. Most of the queries need tagged mode, but when
# we want to run a "p4 job -o" to archive the job, we need it in plain text.
#
# This class is also the reason why P4Table is not thread safe as all of the
# other classes that execute commands use it directly. I wanted to avoid
# excessive passing of P4 objects but in the end I'll probably have to go
# that way.
#

class P4Global

    @@p4t = P4.new
    @@p4u = P4.new
    @@p4t.parse_forms
    @@p4t.exception_level = 1

    #---------------------------------------------------------------------------
    # Set up the Perforce environment
    #---------------------------------------------------------------------------
    
    # Set the username for both tagged and untagged mode
    def P4Global.p4user=( user )
	@@p4t.user = user
	@@p4u.user = user
    end

    # Set the port for both tagged and untagged mode
    def P4Global.p4port=( port )
	@@p4t.port = port
	@@p4u.port = port
    end

    # Set the password for both tagged and untagged mode
    def P4Global.p4passwd=( password )
	@@p4t.password = password
	@@p4u.password = password
    end

    # Set the clientspec for both tagged and untagged mode
    def P4Global.p4client=( client )
	@@p4t.client = client
	@@p4u.client = client
    end

    # Connect both tagged and untagged clients to the server
    def P4Global.connect
	@@p4t.connect
	@@p4u.connect
    end

    # Disconnect both tagged and untagged mode clients
    def P4Global.disconnect
	@@p4t.disconnect
	@@p4u.disconnect
    end
    
    # Returns the tagged mode client
    def P4Global.tagged
	@@p4t
    end

    # Returns the untagged mode client
    def P4Global.plain
	@@p4u
    end

    # Sets the debug level for both tagged and untagged mode (2 is good)
    def P4Global.debug=( level )
	@@p4t.debug = level
	@@p4u.debug = level
    end
end

#
# Class for creating and formatting record id's. Normally these are just
# of the form "<tablename>/<nnnnnn>"
#

class P4RecId

    #---------------------------------------------------------------------------
    # Class methods.
    #---------------------------------------------------------------------------

    #
    # Get the next ID for the specified table using
    #
    # 1. p4 counter <table>
    # 2. p4 counter <table> = <n+1>
    #
    # Yes, this has a race condition, but there's no atomic get and set
    # operation for Perforce counters.
    #
    def P4RecId.next( table )
	p4 = P4Global.tagged
	val = p4.run_counter( table.name ).shift.to_i
	val += 1
	p4.run_counter( table.name, val )
	P4RecId.new( table, val )
    end

    #
    # Instantiate a new P4RecID from the job name ( "<table>/<id>" ). Used
    # to load existing records.
    #
    def P4RecId.new_from_job( job )
	( table, seq ) = job.split( "/" )
	P4RecId.new( P4Table.new( table ), seq )
    end

    #
    # Construct a new RecID for the given table name and sequence number
    #
    def initialize( table, seq )
	@table = table
	if ( seq.kind_of?( String ) )
	    if ( seq =~ /^0*([1-9]\d+)$/ )
		@seq = $1.to_i
	    else
		@seq = seq.to_i
	    end
	else
	    @seq = seq
	end
    end

    #---------------------------------------------------------------------------
    # Instance methods
    #---------------------------------------------------------------------------

    attr_accessor :table, :seq

    #
    # Format the sequence number for printing
    #
    def seq_str
	sprintf( "%06d", @seq )
    end

    #
    # String representation of an id
    #
    def to_s
	@table.name + "/" + seq_str()
    end
end


#
# Class for representing files that are versioned in the depot. Files can
# be either (a) the meta file containing the metadata about the record 
# (basically a "p4 job -o <recid>") or they can be other files which are
# attached to the record.
#
class P4RecFile

    # 
    # Construct a new P4RecFile object based on the supplied filename and 
    # RecId.
    #
    def initialize( name, id )
	@name 		= name
	@id 		= id
	@exists 	= false
	@depot_path 	= nil
	@ws_path 	= nil
    end

    attr_reader :name, :id

    #
    # Convert the depot path into a client path. Obviously this precludes 
    # complex client maps as we don't want to reimplement Perforce mappings
    # in Ruby.
    #
    def ws_path
	return @ws_path if ( @ws_path )

	root = P4Global.tagged.fetch_client[ "Root" ]
	path = depot_path.sub( DEPOT_ROOT_PATH, root )
	@ws_path = path.gsub!( "/", SEP )
    end


    #
    # Explicitly set the depot path for a file. Needed to allow old records
    # to be found in their old locations even if the storage map has since
    # changed.
    #
    def depot_path=( path )
	@depot_path = path
    end

    #
    # Compute the depot path based on the table storage map and the id
    #
    def depot_path
	return @depot_path if @depot_path
	table_path = @id.table.storage_map( @id ).join( "/" )
	@depot_path = [ DEPOT_ROOT_PATH, table_path, @id.seq_str, @name ].join( "/" )
    end

    #
    # Does the file exist?
    #
    def exists?
	@exists
    end

    #
    # Set the existence flag for a file explicitly
    #
    def exists=( bool )
	@exists = bool
    end

    #
    # Open this file for add
    #
    def add
	if ( exists? )
	    raise( RuntimeError, "Can't open existing file for add", caller )
	end
	P4Global.tagged.run_sync
	P4Global.tagged.run_add( ws_path() )
    end

    #
    # Open this file for edit
    #
    def edit
	if ( ! exists? )
	    raise( RuntimeError,"Can't open non-existent file for edit", caller)
	end

	P4Global.tagged.run_sync
	P4Global.tagged.run_edit( depot_path() )
    end

    #
    # Open this file for delete
    #
    def delete
	if ( ! exists? )
	    raise( RuntimeError, "Can't delete non-existent file", caller )
	end

	P4Global.tagged.run_sync
	P4Global.tagged.run_delete( depot_path() )
    end

    #
    # Read the contents of the file (returns a String)
    #
    def read
	P4Global.tagged.run_sync
	f = File.open( ws_path, "r" )
	buf = f.read
	f.close
	return buf
    end

    #
    # Write the contents of the file
    #
    def write( string )
	P4Global.tagged.run_sync
	mkdir
	File.open( ws_path, "w" ) { |f| f.write( string ) }
    end

    #
    # Create the directory for the workspace file
    #
    def mkdir
	P4Global.tagged.run_sync
	dirs = ws_path.split( SEP )
	dirs.pop
	dir = ""
	dirs.each do
	    |d|
	    dir += d + SEP
	    Dir.mkdir( dir ) unless File.directory?( dir )
	end
    end

    #
    # Render the filename as a string (we use depot syntax)
    #
    def to_s
	depot_file
    end
end


#
# Main record manipulation class. Handles the loading and saving of records
# as both jobs and files.
#
class P4Record

    #***************************************************************************
    # Class methods
    #***************************************************************************

    #
    # Load a record from its associated job. Use to load existing records
    # rather than constructing new ones. Raises a RuntimeError if the
    # record does not exist.
    #
    def P4Record.load( id )

	if ( ! P4Record.exists?( id ) )
	    raise( RuntimeError, "Record #{id.to_s} does not exist", caller )
	end

	rec = P4Record.new( id )
	rec.spec = P4Global.tagged.fetch_job( id.to_s )
	if rec.spec.has_key?( "files" )
	    rec.spec[ "files" ].each do
		|file|
		name = file.sub( ".*/", "" ).chomp
		f = P4RecFile.new( name, id ) 
		f.exists = true
		f.depot_path = file.chomp
		rec.files[ name ] =  f
	    end
	end
	rec.exists = true
	return rec
    end

    #
    # Load all records matching a query expression. Uses a dynamically built
    # "p4 jobs -e" expression to identify the matching records and then
    # loads each record. Returns an array of P4Record objects.
    #
    def P4Record.query( table, expr )
	expr = "job=#{table} & ( " + expr + " )"
	P4Global.tagged.run_jobs( "-e", expr ).collect do
	    |job|
	    rec = P4Record.new( P4RecId.new_from_job( job[ "job" ] ) )
	    rec.spec = job
	    rec.spec[ "files" ].each do
		|file|
		name = file.sub( ".*/", "" ).chomp
		f = P4RecFile.new( name, id ) 
		f.exists = true
		f.depot_path = file
		rec.files[ name ] =  f
	    end
	    rec.exists = true
	    rec
	end
    end

    #
    # Create a new record in the given table. Returns the skeletal record
    # populated with the default values from the jobspec for editing.
    #
    def P4Record.create( table )
	id = P4RecId.next( table )
	rec = P4Record.new( id )
	rec.spec = P4Global.tagged.fetch_job( id.to_s )
	return rec
    end

    #
    # Test for record existence
    #
    def P4Record.exists?( id )
	jobs = P4Global.tagged.run_jobs( "-e", "job=" + id.to_s )
	return jobs.length > 0 
    end

    #
    # Constructor: DON'T USE NEW DIRECTLY, CALL create/load.
    #
    def initialize( id )
	@id 	= id
	@table	= id.table
	@seq	= id.seq
	@spec	= nil
	@exists	= false
	@files	= Hash.new

	@files[ "meta" ] = P4RecFile.new( "meta", @id )
    end

    attr_accessor 	:id, :table, :seq, :meta, :spec
    attr_reader		:files

    #
    # Test for record existence
    #
    def exists?
	return @exists
    end

    #
    # Explicitly set record existence (or otherwise)
    #
    def exists=( bool )
	@exists = bool
    end


    #
    # Get a file handle by name (not the depot path). Returns a P4RecFile
    # object.
    #
    def get_file( name )
	return @files[ name ] if @files.has_key?( name )
	raise( RuntimeError, "Record contains no file called #{name}", caller )
    end

    #
    # Get the meta file specifically. Just shorthand.
    #
    def metafile
	get_file( "meta" )
    end

    #
    # Iterate over the files in the record
    #
    def each_file
	@files.each_value { |f| yield( f ) }
    end

    #
    # Add a new file attachment to this record
    #
    def add_file( name )
	if ( name == "meta" )
	    raise( RuntimeError, "The meta file already exists.", caller )
	end

	nfile = P4RecFile.new( name, @id )
	nfile.add
	@files[ "name" ] =  nfile
    end

    #
    # Remove a file attachment
    #
    def rm_file( file )
	if ( file == "meta" )
	    raise( RuntimeError, "You can't delete the meta file", caller )
	end

	f = get_file( file )
	f.delete
	@files.delete( file )
    end

    #
    # Abandon all edits to this record. Use with care: it reverts the
    # files that are open for add/edit/delete etc. but it doesn't
    # reload the record from the job - it may be a new record. You
    # should either discard the record or reload it yourself after 
    # calling abandon().
    #
    def abandon
	p4 = P4Global.tagged
	each_file do
	    |f|
	    fs = p4.run_fstat( f.ws_path )
	    if ( fs && fs[ "action" ] == "add" )
		File.unlink( f.ws_path )
	    end
	    p4.run_revert( f.depot_path )
	end
    end

    #
    # Get the list of files attached to this record in depot syntax
    #
    def file_list
	@files.values.collect do
	    |f|
	    f.depot_path
	end.compact.join( "\n" )
    end


    #
    # Update the record. Updates the job and then archives the job into the
    # meta file and saves any attached files. Provide the description you'd
    # like to see attached to the change
    #
    def save( desc )
	# First rewrite the files list in case it's been modified
	@spec[ "files" ] = file_list

	# Next update the job with the values in the spec
	P4Global.tagged.save_job( @spec )

	# Now update the metafile with the job -o output
	if ( exists? )
	    metafile.edit
	else
	    metafile.add
	end

	metafile.mkdir
	metafile.write( P4Global.plain.fetch_job( @id.to_s ) )

	# Now submit
	change = P4Global.tagged.fetch_change
	change[ "Description" ] = desc
	P4Global.tagged.submit_spec( change )

	@exists = true
    end

    #
    # Delete a record. Deletes the files and the job.
    #
    def delete( desc )
	if ( ! exists? )
	    raise( RuntimeError, "Can't delete. Record doesn't exist.", caller )
	end

	@files.values.each { |f| f.delete }
	change = P4Global.tagged.fetch_change
	change[ "Description" ] = desc
	P4Global.tagged.submit_spec( change )
	P4Global.tagged.run_job( "-d", @spec[ "job" ] )
	@files = Hash.new
	@spec[ "files" ] = Array.new
	@exists = false
    end


    #
    # Obliterate a record. Does exactly what it says on the tin.
    #
    def obliterate
	if ( ! exists? )
	    raise( RuntimeError, "Can't oblit. Record doesn't exist.", caller )
	end

	args = @files.values.collect { |f| f.depot_path } 
	P4Global.tagged.exception_level = 1
	P4Global.tagged.run_sync( args.collect { |a| a += "#none" } )
	P4Global.tagged.exception_level = 2
	P4Global.tagged.run_obliterate( "-y", args )
	P4Global.tagged.run_job( "-d", @spec[ "job" ] )
	@spec[ "files" ] = Array.new
	@files = Hash.new
	@exists = false
    end

    #
    # Allow direct access to the fields in the jobspec by making them virtual
    # method names of the form _<field>()
    #
    def method_missing( meth, *args )
	meth = meth.to_s
	raise if ( meth[0..0] != "_" )
	meth = meth[ 1..-1 ]
	if ( meth =~ /^(.*)=$/ )
	    meth = $1
	    @spec[ meth ] = args.shift
	elsif ( args.length == 0 && @spec.has_key?( meth ) )
	    @spec[ meth ]
	else
	    ""
	end
    end

end


#
# Base class for all table classes. All user defined tables should be derived
# from this class. At the bare minimum you should provide an initialize() 
# that removes the need for the table name to be specified at construction. 
# Something like:
#
#  class MyTable < P4Table
#    def initialize
#      super( "mytable" )
#    end
#  end
#
# You should also make sure that you are handling P4Exceptions to trap and
# report Perforce errors if they occur.
#
class P4Table

    #
    # Construct a new P4Table object for the named table
    #
    def initialize( name )
	@name = name
    end

    attr_reader :name

    #
    # Add a new record to the table. Returns the record for editing. Note
    # the record is not saved until you call P4Record#save
    #
    def new_record
	P4Record.create( self ) 
    end

    #
    # Load an existing record from the repository
    #
    def load_record( seq )
	P4Record.load( P4RecId.new( self, seq ) )
    end

    #
    # Search for matching records.
    #
    def query( expr )
	P4Record.query( name, expr )
    end

    #
    # Return an array of directory components representing the path where
    # the files for this object should be stored. This allows you to tweak
    # the directory structure your files are stored in so you don't end up
    # with 1000's of files in a single directory. 
    # 
    # It is intended that you override this method in your derived classes.
    # Note that you can change the storage map for a table at any time
    # without affecting existing records. This is because the file paths
    # for existing records are stored in the job and only the file paths
    # for new records will be computed using storage_map()
    #
    # For example, to group records by the year and month in which they
    # were created you could use this:
    #
    #   def storage_map( id )
    #     Time.now.to_a[4,2].reverse + [ id.table.name ]
    #   end
    #
    def storage_map( id )
	return [ id.table.name ]
    end
end