<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="GENERATOR" content="Mozilla/4.05 [en] (X11; U; OSF1 V4.0 alpha) [Netscape]">
<meta name="Author" content="Jeff A. Bowles"><title>FAQ for integrating Perforce with builds</title></head>
<body><x-html>
<h1>
FAQ for integrating Perforce with your build scripts</h1>
<p>
Contributed by Jeff Bowles
</p><h3>
Note - the intended audience for this FAQ is a shell/perl hacker who writes
or maintains local build scripts and wants to integrate Perforce into them.</h3>
<ol>
<h5>
<b>Labels</b></h5>
<li>
<a href="#createlabel">How do I create
a label? What should I include in a label?</a></li>
<li>
<a href="#overnightlabels">If I want
to create a new label for each overnight build, how would I do it?</a></li>
<li>
<a href="#labels_vs_changenums">Should I
use labels or change numbers to track the contents of overnight builds?</a></li>
<li>
<a href="#makepatch">So the release
needs to be remade three months after it went out. What do I do?</a></li>
<li>
<a href="#makeofficial">I like this
overnight build and want to officially designate it as a "release" build.
What should I do?</a></li>
<li>
<a href="#releasebuild">I need to create a
release build - what's different in that process?</a></li>
<h5>
<b>Clients</b></h5>
<li>
<a href="#populate_build_area">How do I populate
the build area from a nightly build script? What are the problems that
will come up?</a></li>
<h5>
Updating depot files from a script</h5>
<li>
<a href="#checkin_logs">How do I
update the depot to include a new log, or to include a new line in a logfile?</a></li>
<li>
<a href="#should_I_checkin_logs">Should l check in my log files?
What files might need to be modified and checked in as part of a build?</a></li>
<h5>
Putting a release out</h5>
<li>
<a href="#backup_for_release">What should I back
up when I put a release out?</a></li>
<li>
<a href="#escrow">How do I make
an "escrow" copy of the files (that built the release) for the off-site
vaults?</a></li>
</ol>
<hr>
<ol>
<li>
<a name="createlabel"></a><b>How
do I create a build label? What should I include in a label?</b></li>
<ul>
<li>
Pick a regular format for the label name, e.g. "<i>bld_xxx_1</i>" where
'<i>xxx</i>' is the name of the codeline you're building and '<i>1</i>'
is just a counter, i.e. the first such label. Remember this format - you'll
need it in the next question!</li>
<li>
Make the label itself:</li>
<p><tt><font color="#990000"> p4 label bld_xxx_1</font></tt>
</p>
<p>It'll dump you into an editor, and you can change the default "Views:"
lines from
</p>
<p><tt><font color="#990000"> //depot/...</font></tt>
</p>
<p>to look solely at the codeline you want, e.g.
</p>
<p><tt><font color="#990000"> //depot/xxx/...</font></tt>
</p>
<p>Save/Quit the editor.
</p>
<li>
Populate the label with the file/revision list you want:</li>
<p><tt><font color="#990000"> p4 labelsync -l bld_xxx_1
//depot/xxx/...</font></tt>
</p>
<p><font color="#000000">or</font>
</p>
<p><tt><font color="#990000"> p4 labelsync -l bld_xxx_1
//depot/...</font></tt>
</p>
<p>In this second example, it's okay to include all files in the depot
- it'll really only include files described in the label spec you edited
when you ran "<tt><font color="#990000">p4 label</font></tt>".
<br>
</p>
<li>
<b>Always include all files [that are under source management] that are
needed to recreate the build</b> . If your build scripts aren't in codeline
"xxx" you'll want to have the label specification include the directory
where they are, and "<tt><font color="#990000">p4 labelsync</font></tt>"
needs to include them.</li>
</ul>
<hr>
<li>
<a name="overnightlabels"></a><b>If
I want to create a new label for each overnight build, how would I do it?</b></li>
<br>
<ol>
<li>
Make sure you're using a regular name for the label, e.g. "bld_xxx_yyy"
where "xxx" is the codeline name and "yyy" is a counter.</li>
<li>
Create the first label in the sequence, which would be "bld_xxx_1". See
the previous question for that.</li>
<li>
In the Unix shell, you would do the following (cut and paste):</li>
<p><tt><font color="#990000"> CodeLineName=xxx</font></tt>
<br><tt><font color="#990000"> LabelSpec=tst_${CodeLineName}_</font></tt>
<br><tt><font color="#990000"> toplabelnumber=`p4 labels |
grep $LabelSpec | sed "s/^Label $LabelSpec//" | \</font></tt>
<br><tt><font color="#990000">
sed "s/ .*//" | sort -r -n | head -1`</font></tt>
<br><i><tt><font color="#990000"> # at this point, we know
the most recent label in this codeline and just need</font></tt></i>
<br><i><tt><font color="#990000"> # to create a new label and
populate it.</font></tt></i>
<br><tt><font color="#990000"> TopLabel=$LabelSpec$toplabelnumber</font></tt>
<br><tt><font color="#990000"> echo Current Label is $LabelSpec$toplabelnumber</font></tt>
<br><tt><font color="#990000"> if [ "$toplabelnumber" != ""
]</font></tt>
<br><tt><font color="#990000"> then</font></tt>
<br><tt><font color="#990000">
newlabelnumber=`expr $toplabelnumber + 1`</font></tt>
<br><tt><font color="#990000">
NewLabel=$LabelSpec$newlabelnumber</font></tt>
<br><tt><font color="#990000">
echo "New Label is $NewLabel"</font></tt>
<br>
<br><tt> <i><font color="#990000">#
the following two commands are from the previous FAQ item.</font></i></tt>
<br><tt><font color="#990000">
<u>p4 label -o -t $TopLabel $NewLabel | p4 label -i</u></font></tt>
<br><tt><font color="#990000">
p4 labelsync -l $NewLabel //depot/...</font></tt>
<br><tt><font color="#990000">
<i># It's best to obsolete labels that are kinda of old, so we</i></font></tt>
<br><i><tt><font color="#990000">
# do it here.</font></tt></i>
<br><tt><font color="#990000">
if [ "$toplabelnumber" -gt 10 ]</font></tt>
<br><tt><font color="#990000">
then</font></tt>
<br><tt><font color="#990000">
obsoletelabelnumber=`expr $toplabelnumber - 10`</font></tt>
<br><tt><font color="#990000">
ObsoleteLabel=$LabelSpec$obsoletelabelnumber</font></tt>
<br><tt><font color="#990000">
p4 label -d $ObsoleteLabel</font></tt>
<br><tt><font color="#990000">
fi</font></tt>
<br><tt><font color="#990000"> else</font></tt>
<br><tt><font color="#990000">
echo "There was no label to match!" 1>&2</font></tt>
<br><tt><font color="#990000">
exit 1</font></tt>
<br><tt><font color="#990000"> fi</font></tt>
<br>
</p>
<li>
Perl hackers will look at the previous "Bourne shell" example and use it
as an example of why you don't want to do string processing in "Bourne
shell". One perl treatment of this follows:
</li>
<tt><font color="#990000"> $CodeLineName = "xxx";</font></tt><br>
<tt><font color="#990000"> $LabelSpec="tst_" . $CodeLineName
. "_";</font></tt>
<p><tt><font color="#990000"> @AllLabels = `p4 labels`;</font></tt>
<br><tt><font color="#990000"> @ThisCodeLineLabels = sort(sortrevlabel
grep(/$LabelSpec/, @AllLabels));</font></tt>
<br><tt><font color="#990000"> $TopLabelLine = $ThisCodeLineLabels[0];</font></tt>
<br><tt><font color="#990000"> $TopLabel = $1 if ($TopLabelLine
=~ /^Label\s(\S+)\s/);</font></tt>
<br><tt><font color="#990000"> $TopLabelNumber = $1 if ($TopLabel
=~ /^\S+_(\d+)$/);</font></tt>
</p>
<p><tt><font color="#990000"> print "The last label in this
codeline is $TopLabel\n";</font></tt>
<br><tt><font color="#990000"> $NewLabelNumber = $TopLabelNumber
+ 1;</font></tt>
<br><tt><font color="#990000"> $NewLabel = "$LabelSpec$NewLabelNumber";</font></tt>
<br><tt><font color="#990000"> print "New Label is $NewLabel\n";</font></tt>
<br><tt><font color="#990000"> system("<u>p4 label -o -t $TopLabel
$NewLabel | p4 label -i</u>");</font></tt> <i># in real life, return code would be checked!</i>
<br><tt><font color="#990000"> system("p4 labelsync -l $NewLabel
//depot/...");</font></tt>
<br><tt><font color="#990000"> if ($TopLabelNumber > 10 )</font></tt>
<br><tt><font color="#990000"> {</font></tt>
<br><tt><font color="#990000">
$ObsoleteLabelNumber = $TopLabelNumber - 10;</font></tt>
<br><tt><font color="#990000">
$ObsoleteLabel = "$LabelSpec$ObsoleteLabelNumber";</font></tt>
<br><tt><font color="#990000">
system("p4 label -d $ObsoleteLabel");</font></tt>
<br><tt><font color="#990000"> }</font></tt>
</p>
<p><tt><font color="#990000"> sub sortrevlabel {</font></tt>
<br><tt><font color="#990000">
my($labelnum_a, $labelnum_b) = ();</font></tt>
<br><tt><font color="#990000">
$labelnum_a = $1 if ($a =~ /.*_(\d+)/);</font></tt>
<br><tt><font color="#990000">
$labelnum_b = $1 if ($b =~ /.*_(\d+)/);</font></tt>
<br><tt><font color="#990000">
return ($labelnum_b <=> $labelnum_a);</font></tt>
<br><tt><font color="#990000">
}</font></tt>
</p>
<li>
<font color="#000000">The underlined part in the above fragments probably
should have a "sed" command between the two 'p4 label' commands, to change
the description for each label.</font></li>
</ol>
<hr>
<li>
<a name="labels_vs_changenums"></a><b>Should
I use labels or change numbers to track the contents of overnight builds?</b></li>
<br>
<ul>
<li>
There's no right answer. The command</li>
<p><tt><font color="#990000"> p4 changes -m1 //depot/xxx/...</font></tt>
</p>
<p>will tell a build script what the top change number for codeline 'xxx'
(assuming it's mapped there), so it can parse this output and use that
in build logs and build e-mail. The developer would type:
</p>
<p><tt><font color="#990000"> p4 sync @12345</font></tt>
</p>
<p>to pull over all files current as of change #12345, but that has a problem:
it'll pull over all files [that his/her client maps in] as of that revision,
not just the files for codeline 'xxx'. You can tell them to type:
</p>
<p><tt><font color="#990000"> p4 sync //depot/xxx/...@12345</font></tt>
</p>
<p>but, by then, you've lost the simplicity that using the change numbers
might've given you.
<br>
</p>
<li>
You should probably always use labels for release builds - when you create
a release you'll often need to include an additional change to one particular
file that you need to track, and you want the label to include <u>exactly</u>
the files used to build the release on the build client.</li>
</ul>
<hr>
<li>
<a name="makepatch"></a><b>So the
release needs to be remade three months after it went out. What do I do?</b></li>
<dl>You'll need a slightly-hacked version of your "get the source and do
a build" script. The hacks are:
<ul>
<li>
It shouldn't use or touch any labels except for the release label you've
remembered to create when you made the release. Delete the lines in the
script that create labels or run "p4 labelsync" on them.</li>
<li>
Change the line in the script that does the "<tt><font color="#990000">p4
sync -f ...#have</font></tt>" so that it looks like:</li>
<p><tt><font color="#990000"> p4 sync -f //depot/xxx/...@labelname</font></tt>
</p>
<p>where "xxx" is the name of the codeline, and "labelname" is the release
label.
</p>
<li>
Bug your system administrators about whether compiler versions, OS releases,
etc, have changed since the release was created. Listen carefully to their
answers.</li>
<li>
Run the script. It should create the original release faithfully, although
most compilers embed timestamps within the .exe and .obj files, so you
might not have byte-for-byte identical files.</li>
<li>
Rerun your regressions to gain confidence in the newly-built product.</li>
</ul>
</dl>
<hr>
<li>
<a name="makeofficial"></a><b>I like
this overnight build and want to officially designate it as a "release"
build. What should I do?</b></li>
<dl>See the previous step - you just need to run "p4 label" (to create
a label) and then "p4 labelsync" to the overnight label or change number
that created the build:
<p><tt><font color="#990000"> p4 labelsync -l release_label_name
//depot/xxx/...@bld_xxx_23</font></tt>
</p><p>or
</p><p><tt><font color="#990000"> p4 labelsync -l release_label_name
//depot/xxx/...@12345</font></tt>
</p><p>The first case is for "overnight build #23 for codeline xxx", if you're
using that convention for overnight builds; the second case is if you use
change numbers to track the overnight build contents.
<br> </p></dl>
<hr>
<li>
<a name="releasebuild"></a><b>I need to create
a release build - what's different in that process?</b></li>
<br>
<dl>This depends on what packages the release for deployment. If you use
the same build mechanisms for overnight and release builds, you should
only need to do a few steps:
<ul>
<li>
Run the overnight build script to create the build.</li>
<li>
Label the build area that created the release, using "p4 label" and then
"p4 labelsync". <i>Pick the 'view' for the label carefully, so that it
doesn't include parts of the depot not included in this release.</i></li>
<li>
Run "p4 label" a second time on the new label, and change the status of
the label to "locked" so that "p4 labelsync" cannot be run against it accidently.</li>
</ul>
</dl>
<hr>
<li>
<a name="populate_build_area"></a><b>How do
I populate the build area for an overnight build? What are the problems
that will come up?</b></li>
<br>
<ul>
<li>
If the Perforce client root is C:/p4root, and the codeline is C:/p4root/xxx,
then you'll need to do 2-3 things to populate the area prior to building:</li>
<ul>
<li>
Make a copy of the scripts that'll do the populating, if there's a chance
that you'll delete them in the next two steps! (Leave a place for your
scripts to stand.)</li>
<li>
Run the OS command to delete the codeline directory, e.g.</li>
<p><tt><font color="#990000"> del/s c:\p4root\xxx</font></tt>
</p>
<p>it's important to do this if you want a clean build - don't rely on
incremental compile for overnight compiles unless the compilation process
takes many hours.
</p>
<li>
Repopulate the source:</li>
<p><tt><font color="#990000"> p4 sync -f C:/p4root/xxx/...#have</font></tt>
</p>
<li>
Update the source to the most recent revisions:</li>
<p><tt><font color="#990000"> p4 sync C:/p4root/xxx/...</font></tt>
</p>
<li>
<font color="#000000">Update your build labels (or remember the current
change numbers) to reflect this new content.</font></li>
</ul>
Deleting the entire codeline build tree (and any "obj" or "lib" directories
if they're not directly under C:\p4root\xxx) is important because old object
files often can bite you.
<br>
<li>
Most problems that come up in this area have to do with obsolete object/library
files. <u>Entirely removing the source area and repopulating saves you
from this headache completely.</u></li>
</ul>
<hr>
<li>
<a name="checkin_logs"></a><b>How
do I update the depot to include a new log, or to include a new line in
a logfile?</b></li>
<br>
<ul>
<li>
To include a new log or edit a file, you should:</li>
<ul>
<li>
Open the file for either "add" or "edit";</li>
<li>
Modify the file in whatever ways make sense;</li>
<li>
Use "p4 change -o" and "p4 submit -i" to create the submitted change.</li>
</ul>
An example of this, in a Unix shell script, follows:
<p><tt><font color="#990000"> operation=add</font></tt>
<br><tt><font color="#990000"> filename=newfile.txt</font></tt>
<br><tt><font color="#990000"> <u>filetype=ltext</u></font></tt>
<br><tt><font color="#990000"> [ -f $filename ] &&
operation=edit</font></tt>
<br><tt><font color="#990000"> description="log of build for
`date`"</font></tt>
</p>
<p><tt><font color="#990000"> p4 $operation -t $filetype $filename</font></tt>
</p>
<p><tt><font color="#990000"> echo New information as of `date`
>> $filename</font></tt>
<br><tt><font color="#990000"> <i># this point, we need to
construct the change submission with</i></font></tt>
<br><i><tt><font color="#990000"> # an accurate description
in the change text.</font></tt></i>
<br><tt><font color="#990000"> p4 change -o > xx.tmp</font></tt>
<br><tt><font color="#990000"> (</font></tt>
<br><tt><font color="#990000">
echo "/^Files:/+1,\$v/$filename/d"</font></tt>
<br><tt><font color="#990000">
echo "/<enter description here>/s/.*/ $description/"</font></tt>
<br><tt><font color="#990000">
echo w</font></tt>
<br><tt><font color="#990000">
echo q</font></tt>
<br><tt><font color="#990000"> ) | ed - xx.tmp</font></tt>
<br><tt><font color="#990000"> p4 submit -i < xx.tmp</font></tt>
<br><tt><font color="#990000"> rm xxx.tmp</font></tt>
</p>
<p><font color="#000000"><i>Perl</i> hackers won't see substantial gains
by doing this in <i>perl</i>, and the 'ed' is actually easier from the
shell script.</font>
<br>
</p>
<li>
<font color="#000000">The underlined part above is important, if the file
you're working with is a giant log file, because giant text files should
include this <u>hint</u> so that the files will be stored in more efficient
ways.</font></li>
</ul>
<hr>
<li>
<a name="should_I_checkin_logs"></a><b>Should l check in my
log files? What files might need to be modified and checked in as part
of a build?</b></li>
<br>
<ul>
<li>
There's no good answer for this - log files are usually huge, and can start
eating space quickly. It's probably best to keep them in a disk area that's
backed up regularly, but not to check them in if you can avoid it.</li>
<li>
If, however, you track overnight builds based on change numbers instead
of labels, you might want to have a file somewhere in the <i>depot</i>
that contains one-line entries of the form:</li>
<br>
<tt><font color="#990000"> change 12345 - overnight build
for 07/22/98</font></tt><br>
<tt><font color="#990000"> change 12366 - overnight build
for 07/23/98</font></tt><br>
and so on. That should be modified (appended to) by the build scripts
using code similar to the example above. <u>It gets really hard to track
which change corresponded to which nightly build, otherwise.</u><br>
</ul>
<hr>
<li>
<a name="backup_for_release"></a><b>What should
I back up when I put a release out?</b></li>
<br>
<br>
<ul>
<li>As a minimum, you'll need to back up the source files and build scripts
in Perforce that created the product. This makes several assumptions:
Usually, however, <u>you'll probably just want to back up the entire Perforce
depot so that you have a complete snapshot of everything</u>, and then
if you restore from this backup you can refer to current labels such as
the <i>product release label</i> you made when you released the product.
(You can also refer to intermediate build labels if you choose, since they'll
be included in the backup.)
<i><font color="#000000">The second and third steps are slight overkill
for many situations, but if you wanna be incredibly paranoid, that's the
approach to take.</font></i>
</li>
<li>
That the build environment (machine, OS, compilers, etc) are easily recreateable
from media you have in-hand;</li>
<li>
That, if you only back up the top revision of those files, you won't need
file revision history and the like.</li>
<li>To make the most comprehensive backup, you'll need to do the following:
</li>
<li>
Checkpoint the Perforce database and save the checkpoint and the subdirectory
$P4ROOT/depot with all its contents:</li>
<p><tt><font color="#990000"> cd $P4ROOT</font></tt>
<br><tt><font color="#990000"> p4d -r
. -jc</font></tt>
</p>
<p><font color="#000000">Note that this is documented in the "FAQ for System
Administrators" and also on-line in the Perforce manuals.</font>
</p>
<li>
<font color="#000000">Save a copy of the Perforce executables you're using,
server and client versions.</font></li>
<li>
<font color="#000000">Do a full system backup of the build machine on which
the product was created. This isn't so much to get the source onto media
as to get compilers, OS, etc.</font></li>
<li>
<font color="#000000">Put this all onto a medium you trust, that'll be
around for a while.</font></li>
<li>
<font color="#000000">Write the <i>product release label,</i> the one that
you created when you built the product, on the media case. Include with
the case a hard-copy printout of the exact commands you typed to create
the media, e.g. <i>created using "tar rvf".</i></font></li>
</ul>
<hr>
<li>
<a name="escrow"></a><b>How do
I make an "escrow" copy of the files (that built the release) for the off-site
vaults?</b></li>
<br>
<br>You can use a label to create an initial fileset that you'll archive.
It's pretty easy:
<ul>
<li>
Create a label using <tt><font color="#990000">p4 label, <i>escrowlabelname1.0,
</i></font></tt>that maps in only the directories you want. (You'll edit
the label specification to either include as many directories as you choose,
or perhaps the exact file list itself.)</li>
<li>
Populate the label, using:</li>
<p><tt><font color="#990000">
p4 labelsync -l <i>escrowlabelname1.0</i> //buildclientname/...#have</font></tt>
</p>
<p>or,
</p>
<p><tt><font color="#990000">
p4 labelsync -l <i>escrowlabelname1.0 </i> @release_source_label</font></tt>
</p>
<p>The first example relies on the build client having the exact files,
at this moment, that it did when it created the build; the second example
relies on having a label of all the source files that were built to create
the release.
</p>
<li>
Create a new Perforce client that maps its client area to an empty part
of the disk, and on that new client run the command:</li>
<br>
<tt><font color="#990000">
p4 sync @<i>escrowlabelname1.0</i></font></tt>
<li>
Back up this new area.</li>
<li>
Run <tt><font color="#990000">p4 label <i>escrowlabel1.0</i> </font></tt>
and mark the label "locked". This will prevent accidently updates of that
label.</li>
<li>
When release 2.0 goes out and you need to do this again, follow the same
steps. When you create the label, however, use the original (release 1.0)
escrowlabel as a template, using<tt><font color="#990000"> p4
label -t <i>escrowlabel1</i>.0 </font></tt> escrowlabel2.0 .</li>
</ul>
</ol>
</x-html><x-html>
<i>Last modified on 7 November 2003.</i>
</x-html></body></html>
| # | Change | User | Description | Committed | |
|---|---|---|---|---|---|
| #1 | 5415 | michael |
Move user FAQ to //guest branch and provide pointers. Remove old branching FAQ. Update page to reference Perforce Software documentation. |
||
| //public/perforce/faq/build.html | |||||
| #7 | 3870 | Jeff Bowles | Cleaning up formatting. | ||
| #6 | 3868 | Jeff Bowles | cleaning up a lot of bullet lists. | ||
| #5 | 1959 | Jeff Bowles | Minor hacks to clean up links. | ||
| #4 | 112 | Laura Wingerd |
Publish "Branching FAQ" (pull in changes from //guest/laura_wingerd/perforce/faq/... ) |
||
| #3 | 27 | Jeff Bowles | Adding important comment. | ||
| #2 | 20 | Jeff Bowles | Cleaning up numbered lists. | ||
| #1 | 1 | laura |
Add FAQs. (Still needs index page.) |
||