Jam Case Study

This article describes an example implementation of Jam by Robert Cowham of Vaccaperna Systems Ltd.

Jam is the freeware Make replacement provided by Perforce.

The customer had a system consisting of over 150 executables built from several thousand source files (originally on OpenVMS but subsequently ported to Unix with no changes to the Jamfiles). A complete build of the system took 4 hours or more.

The challenge was to have a set of build scripts to build the complete system, and yet also make it easy for individual developers to build and test parts of the system (e.g. just a few executables). We didn't want the developers to require a copy of the whole system in their workspace just to build a small part of it, and any build should be available in minutes rather than hours!

For this system some new Jam rules (in the Jambase) were required to build:

• Cobol and C files with embedded SQL
• Cobol header processing ("copy" statements) to automatically pick up dependencies similar to #include processing
• Generation of Cobol shared copy libraries
• Ingres 4GL Forms preprocessing and compilation

We had a complete (read-only) reference copy of the system source code with associated Jamfiles synchronised from Perforce in a central directory. Note that keeping this tree up-to-date was a manual process outside the scope of this document.

All the developers had to do was set an environment variable to point to this tree, and then set the equivalent of search paths so that files in their current directory were preferred to files in the reference copy of the system, and yet if a file was not found in the current directory it would be taken from the reference tree. This included the complete hierarchy of Jamfiles so that developers always built things appropriately and didn't forget to link needed libraries etc.

Main System

The main build system was a standard multilevel Jam system with 3 files at the top:

• Jamfile        - top level Jamfile - includes subsystem Jamfiles
• Jamrules    - global rules for this system
• Jambase    - modified Jambase

Main Jamfile

# This is the main Jam file for building ABS system.
# Starts at the top level and causes appropriate builds in
# sub directories.

# Set up our definitions

actions Initialise
{
    show logical globals
    show logical wscopylib
    show logical top
    show logical curjamdir
}

# Ensure above rule is called
Initialise first ;
ALWAYS first ;

# Define Jam variable as a VMS directory (needs this sort of syntax)
CURJAMDIR = CURJAMDIR: ;

# Point Jam var to root of Reference Tree
ABS = TOP: ;

# Include the system Jamrules file
ABSRULES = TOP:[000000]Jamrules ;

SubDir ABS ;

SubInclude ABS common ;
SubInclude ABS cos ;
etc...
SubInclude ABS utils ;

# Dependencies on all the source files the ABF image depends on
# Note that the first dependency is the database to use.
# All the images dependencies are declared in the subsystem files using
# ImageSources.
Image abs : $(ABFDATABASE)
        ;

Contents of a Subsystem Jamfile

There is a Jamfile for each subsystem, which lives in the appropriate subsystem directory.

The jamfile for a subsystem (DAQ) has a structure similar to:

# Jamfile for the DAQ subsystem
SubDir ABS daq ;
SubDirHdrs $(ABS) common ;

# Add searches to other subsystem directories
SubDir ABS daq ;
SubDirHdrs $(ABS) common ;

# Add searches to other subsystem directories
SubDirSearch $(ABS) utils ;
SubDirSearch $(ABS) cos ;

The next line specifies a particular compiler flag in order to avoid link errors when building this module.

# Special flag for C RTL linking on this module to avoid link error.
ObjectCcFlags daq1801b11.sc : /prefix_library_entries=all_entries ;

The next rule defines that the library whose name is given by the DAQLIB variable (defined in the top level Jamrules file), depends on the specified source files.

Library $(DAQLIB) :
    daq0201s02.scb
    coboldaq0201s02.sc
    daq0201s03.scb
    ;

The next rule gives dependencies of the abs.exe image produced via ABF. This rule only includes sources from this subsystem.

# Dependencies on all the source files the ABF image depends 
# on in this subsystem
ImageSources abs : $(ABFDATABASE)
    coboldaq0201s02.sc
    daq0201s.frm
    daq0201s.osl
    daq0201s01.scb
    ;

Then we specify executables in this subsystem with appropriate link libraries, e.g.:

LinkLibraries daq0203b : $(DAQLIB) $(UTILSLIB) ;
Main daq0203b :
        daq0203b.scb
        daq0202s01.scb
        daq0203b01.scb
        ;

Building only part of the System

Let us assume the following:

1. You are working in directory DEV_USER:[fred.work].
2. You want to build a debug version of a new module crc0508r.exe
   with new globals gl_max_no_plan_in_crc0508r and
   gl_max_no_data_in_crc0508r.
3. You have used Perforce to check out the files that you want to
   change into your working directory.
4. You must create new text file gl_max_no_plan_in_crc0508r.txt and
   gl_max_no_data_in_crc0508r.txt and 'p4 add' them to Perforce.
5. Remember, when all of the changes are complete the Perforce
   Jamfile in src/common must be updated and submitted with the two new
   globals.

Using the method described in this section you can build the executable crc0508r.exe with /debug in your local directory using
released sources, and the released jamfile definition of which modules it contains. Any sources found in your local working directory will be used in preference to the ones in the release hierarchy. The resulting executable will be built in your [.exe] sub-directory.

In this case all you need in your local directory are:

• jamfile
• jam.com
• crc0508r.txt
• gl_max_no_plan_in_crc0508r.txt
• gl_max_no_data_in_crc0508r.txt

All other source files will be found in the reference directory tree.

Developer Jamfile

When working on smaller parts of the system, and in particular if you want to build with /debug switched on, you are better off creating a separate directory with a Jamfile and building in that directory. You can still tell Jam to go and find the relevant sources in the reference directory tree, yet build the objects and executables in the current directory.

# This jamfile is provided to hook into the Main Jam hierarchy for a release.
# It requires two environment variables to be set up:
#
# CURJAMDIR     - The current directory
# TOP           - The Root of the reference Release hierarchy
#
# E.g.
# define curjamdir USER:[XLRC1.JAMTEST]
# define /translation=concealed top $disk11:[build.p4.]

actions Initialise
{
    show logical globals
    show logical wscopylib
    show logical top
    show logical curjamdir
}

# Ensure above rule is called
Initialise first ;
ALWAYS first ;

# Define Jam variable as a VMS directory (needs this sort of syntax)
CURJAMDIR = CURJAMDIR: ;

# Point Jam var to root of Reference Tree
ABS = TOP: ;

# Include the system Jamrules file
ABSRULES = TOP:[000000]Jamrules ;

# From here on the Jamfile is the same as the project one.
SubDir ABS ;

SubInclude ABS common ;
SubInclude ABS cos ;
SubInclude ABS utils ;

# Dependencies on all the source files the ABF image depends on
# Note that the first dependency is the database to use.
# All the images dependencies are declared in the subsystem files using
# ImageSources.
Image abs : $(ABFDATABASE)
        ;

# Add the rules for the new executable. If the executable already exists
# then it is not necessary to add anything here as the rules will be 
# picked up from the jamfile in the release structure.
LinkLibraries crc0508r : $(UTILSLIB) $(CRCLIB) $(PARLIB)  ;
Main crc0508r :
        crc0508r.scb
        crc0502r.scb
        crc0504r01.scb
        ;

Note that any resulting object, library and executable files will be built in directory [.exe] i.e. USER:[fred.work.exe].

Jam.com - Command

This is a VMS command file used to execute the build. The reason for this is that it sets some environment variables and can be easily spawned to a batch queue.

This is a slightly modified version of the jam.com file used to build the complete system release, but it is adapted to setup appropriate local variables.

$! jam.com
$!<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$! MAKE MODIFICATIONS HERE (if you move this file).
$!
$! Define the root of the reference Directory Tree in TOP
$ releasedir = "newbuild:[newbuild]"
$! Define the base directory which we are working in.
$ rootdir = "user:[robert.test]"
$
$! Define the database to be used when building ABF images - only needed for abs.exe
$! BE VERY CAREFUL NOT TO USE SOMEONE ELSES DATABASE BY ACCIDENT!!!!
$! Leave as dummy definition if not needed.
$ abfdatabase = "dummydb"
$
$!>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
$! NO MODIFICATIONS BETWEEN HERE AND THE LAST SECTION
$!

$!******************************************************************
$! Finally get around to building - note the specification of the database.
$!
$! Use "-sOPTIM=/debug/noopt" for debug compilation
$! Use "-sLINKFLAGS=/debug" for debug linking
$! Note that we include wscopylib and globals.
$!
$! Examples are:
$! jam "-sABFDATABASE=''abfdatabase'" abs.exe
$! jam "-sABFDATABASE=''abfdatabase'" "-sOPTIM=/debug/noopt" -
    "-sLINKFLAGS=/debug" wscopylib.tlb daq1701b.exe
$! jam "-sABFDATABASE=''abfdatabase'" wscopylib.tlb globals.tlb daq1601b.exe
$!
$!<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
$! YOU CAN MAKE MORE MODIFICATIONS HERE
$
$ jam  "-sABFDATABASE=''abfdatabase'" "-sOPTIM=/debug/noopt" -
    "-sLINKFLAGS=/map/debug" -
    globals.tlb wscopylib.tlb daq1601b.exe 

(Note the line continuation characters on VMS)

Changes to Jambase

The following changes to the Jambase were made:

Top