Perforce FAQ for System Administrators
Contributed by Jeff Bowles
Note - the intended audience for this FAQ is a system administrator who
knows some Perforce commands as a user, but needs more information to be
able to do some Perforce administration tasks such as setting up a new
Perforce user or dealing with restarting the Perforce server after a recovery
from a catastrophic disk crash.
This FAQ assumes that there is a Perforce Administrator who knows some
of the more complex parts of Perforce, who can provide help, counseling,
and deal with policy-specific issues....
Client Stuff
Non-admin Usage Stuff
Policy Stuff
Admin Issues (backups et al)
-
What's
a client? What's a client spec? How do I tell what this client spec does
and doesn't do? How do I add a new client?
Glossary
-
depot - the term usually applied to the Perforce repository, which
lives on the other end of the network connection specified with $P4PORT.
There's also a syntax used to specify the path names of files in the
depot: //depot/x/y/z/file.java.
-
client - the [local] disk area on a developer's machine, onto which
read-only copies of the source files are copied. Usually there's one client
per user and one client per machine, but that's completely up to the user
and to the people who set it up. (The command p4 client allows you to change
the mapping so that a file is stored, locally, to the directory/file you
specify. See the next item for details.)
-
client specification - documented here,
the internal information specific to a Perforce client, that specifies
the following:
-
The date/time the client was created.
-
The local directory name that's the top of the local copy of the file set
retrieved from the depot.
Root: D:/weblogic
-
An owner, which isn't that useful.
-
Lines at the end that look like
//depot/x/y/... //clientname/x/y/...
//depot/x/z/%1 //clientname/x/old-z/%1
-
The way to read this is "left is the depot pathname and right is the
local disk pathname". These lines are the meat of the client specification
and can be a little tricky.
-
The first line says "all files in depot directory
/x/y on the depot should be installed in D:/weblogic/x/y, preserving the
path names below x/y" (for this example). Every "..."
is a wild card meaning "everything from here down".
-
the second lines says "the files in depot directory
/x/z (but no subdirectories in it) should be put into D:/weblogic/x/old-z".
-
The default client spec maps all depot files to the local disk, using:
//depot/... //clientname/...
-
Usually we'll make a client with the "default client specification" and
use that to create new clients. For example, if the a client named XXX
exists and has the perfect client spec for a new client to imitate, then
P4CLIENT=newclientname
p4 client -t XXX
will
create the new client using the existing "XXX" client as a template. (Hint:
p4 client
uses the current directory as the default for the Root: of the local file
set it'll retrieve, so you can save keystrokes by chdir'ing
there before running p4 client.)
-
client software - the copy of p4.exe
or p4 ("client") that someone's downloaded
from Perforce, Inc. that connects
to a database process ("server") that administers the depot.
There are many platforms supported, which means there are many different
machines for which you can download a new p4
client executable. [Aside: there are no legal restrictions at all w.r.t.
Perforce client software - the licenses are based on the number of
legal users, and we can download client software for as many different
platform types as we choose.]
How do I tell what this client spec does and doesn't
do?
-
p4 client -o clientname
will tell you a lot about a remote client. The two items to stare at are
"Root:" (where
the local files are loaded) and "View:"
(the mapping from depot-name to local-name).
-
Most client-spec headaches have to do with the
view not including a new branch/codeline recently installed - your
build engineers can usually alert you these problems - and problems with
wild cards in a "View:" line. ("..." and "%1", above, are both such wild
cards and are documented here.
In general, your build engineers and/or your local Perforce administrators
should have a handle on this.)
How do I add a new client?
-
See the item, above, that talks about p4
client -t XXX.
-
How
do you add/delete files? Can you retrieve a deleted file? How?
-
To add a file, run p4 add filename;
to delete a file, run p4 delete filename.
In both cases, you've really just opened the file for add/delete and
need to run p4 submit afterwards to get it
checked into the depot.
-
To retrieve a deleted file:
-
You need to know the filename in the form:
//depot/dev/src/sandbox/jab/GNUdepends.def
You'll need some history information, so run the
following command:
D:\weblogic\dev\src>p4 filelog //depot/dev/src/sandbox/jab/GNUdepends.def
//depot/dev/src/sandbox/jab/GNUdepends.def
... #4 change 19737 delete on 1998/05/05 by jab@jab.greenwich 'Change to move jab's erro'
... ... delete into //depot/dev/src030100b01/sandbox/jab/GNUdepends.def#2
... #3 change 19611 edit on 1998/05/04 by jab@jab.sol 'Bits of hacking to support stag'
... ... branch into //depot/dev/sandbox/jab/GNUdepends.def#1
... ... branch into //depot/dev/src030100b01/sandbox/jab/GNUdepends.def#1
In this case, revision #4 deleted the file,
and revision #3 was the last revision that had "real" contents.
A fast way to look at the contents of revision #3
of this file:
p4 print //depot/dev/src/sandbox/jab/GNUdepends.def#3
Note that the first line of output is a line of the
form " //depot/dev/src/sandbox/jab/GNUdepends.def#3".
If you don't want that, run p4
print -q instead
of p4 print.
To retrieve the contents and check 'em in again,
you need to "re-add" the file:
p4 print -q GNUdepends.def#3 > newfile
p4 add GNUdepends.def
cp newfile GNUdepends.def
p4 submit
(There's two other ways to do this: one is to branch
revision #3 of the GNUdepends.def, creating a new version, but unfortunately
it would need a different name; another would be to "p4 sync -f GNUdepends.def#3"
and then move that copy to the side, then do the "p4 add" shown above.)
The unasked question is Can I expunge a file that
I checked into the wrong place? That question is answered in the
Perforce documentation, under "p4 obliterate", but this shouldn't be done
without your Perforce administrator standing over you. (That operation
is completely irreversible.)
-
How
does Perforce deal with CR/LF issues? Can this be overridden?
Background
The machine on which the "p4" client software is run dictates the CR/LF
behavior. If you extract the files from the Perforce depot using the Windows
version of "p4" it'll have CR+LF as line separators for text files,
even if you physically wrote the files to a Unix disk that was remotely
mounted using NFS or Samba. (This means that the Macintosh version of the
"p4" client uses CR for end-of-line, always, and the Unix client uses LF
for end-of-line.)
How do you tell if a file is "text" or not?
If a file's checked in, run p4 file XXX
where XXX is the name of the file. You'll get output of the form:
//depot/dev/src/GNUdepends.def#3 - edit change 20027 (ktext)
//depot/dev/src/GNUmakefile#188 - edit change 21753 (text)
//depot/native/events/EventsGenCtl.bmp#3 - delete change 1103 (binary)
Unless it was specified when the user ran p4 add,
the local Perforce client guessed based on the first few bytes of the file.
(NUL characters and non-ASCII are dead giveaways - if they're present,
the file has to be binary.)
What if you it guessed wrong, and you need to
change the type of a file so that it does end-of-line processing?
You'll need to open the existing file for editing,
change its type, and submit the change:
p4 edit -t text //depot/native/events/EventsGenCtl.bmp
p4 submit
In this case, you'll probably be treating a .bmp
file like it's text, which is pretty silly, but this gets the point across.
You can go back and forth with a file, if you choose - change the type
of a file from text to binary, or binary to text. (The k
in ktext, above, means "expand
keywords like $Id: //depot/internal/staff/jab/P4AdminFAQ.html#18$ in the
contents when it's fetched from the depot. There are other types, also:
p4 help reopen lists 'em. There's
a type for "long text files that are computer-generated such as PDF files"
- be sure to note that one.)
Ways to force it to not interpret end-of-line, to leave the file contents
alone:
Change the file type from text to binary, and make sure that it's byte-for-byte
the exact contents you want.
How do I get Unix end-of-line processing behavior on my Windows Perforce
client?
Short answer: I don't know. Certainly, you can edit the files on Unix,
store what you want there, and change the file type to "binary" to guarantee
that the contents aren't munged when you fetch 'em with the Windows client.
that's probably the fastest, but ugliest way.
-
How
do I tell what some else's client spec looks like? How do I change the
e-mail address for a user is, if it's not the same as their P4 user name?
Looking at someone else's client...
To look at your client, you run:
p4 client
(dumps you into an editor)
or
p4 client -o
(dumps information to standard output)
To look at someone else's client, run:
p4 -c client-name client
or
p4 -c client-name client -o
(Don't change this information (inside the editor)
without eventually informing the user who uses it the most - they could
get cranky about it!)
Looking at someone else's user information...
To look at your user information, you run:
p4 user
(dumps you into an editor)
or
p4 user -o
(dumps information to standard output)
To look at someone else's user information, you run:
p4 user
their-name
(dumps you into an editor)
or
p4 -u their-name user -o
(dumps information to standard output)
(Note that, if it dumped you into an editor, the
file's readonly. To modify another user's information, you will need to
utter a couple of extra commands not included in this FAQ - see your local
Perforce Administrator for those. )
How would I change someone's e-mail address in
the Perforce user database?
Run p4 user
user-name (which
dumps you into an editor) and change it.
Note - if you're trying to automate this...
Most commands that use the option -o
to write to standard output have a -i
option to read from standard input. By doing this, you could write a script
that appends "@weblogic.com" to the e-mail user with the commands:
p4 user -o > tmp-file
sed '/^Email:/s/$/@weblogic.com/' < tmp-file | p4 user -i
The underlined part will need a modification to succeed
- it's deliberately omitted from this FAQ for security reasons. See
your Perforce Adminstrators for details.
(If you do this sorta thing, put a lot more error
checking into the script that you see here!)
-
Should
we store binaries in Perforce? Why, or why not?
Sure, but they can take up some space.
There are several things you need to do when you store a binary in Perforce:
-
Make sure its file type is "binary"
(or "xbinary")
when you check it in. Binary files are stored in a different manner than
text files, because with text files it makes sense to store a simple delta
between two revisions. For certain generated text files (PostScript, Adobe
Acrobat PDF files) and binaries it doesn't make sense, so pick the file
type appropriately.
-
Perforce stores enough information to recreate
each revision of the binary, so if the binary is 100 Mb and you keep
checking in new versions every day, you might run out of disk very quickly.
-
A binary file stored in a branch doesn't immediately
take up more space, but has the potential to fill up each user's client
area by having multiple copies of [potentially] big files. [This can
happen because a user might have multiple branches mapped to their disk,
and they'd get a separate copy of the file for each branch.]
The main headache with storing binaries is that it's
tempting to check in the complete release you developed using Perforce,
so that "x.c" and "x.h" and "x.o" and "x.out" all appear in Perforce. That
might not be the strategy you want, partly because it introduces an ambiguity:
someone might update "x.c" or "x.h" but not realize that "x.out" is getting
shipped, but wasn't updated. (That's dangerous.)
Addendum: a recent version of Perforce supports "compressed files" and
can save a lot of space on the depot when used. If you need to store binaries
in Perforce, it's handy to research this new feature.
Also, there's a new file type called "tempobj" which stores "temporary
object" files. It's a good way to check in the most recent revision of
some object file, obsoleting all prior revisions, so that developers will
be able to use this object (or library or executable) without recompiling
it.
-
On Windows/NT, how
can I look at internal variables that the Perforce client uses to connect?
Environment
There are the environment variables described
here, but the most frequently used are $P4PORT and $P4CLIENT.
Internal variables
In addition, p4 set shows some entries hidden
[in the Windows registry, I believe] that can be set using
p4 set P4PORT=p4server:6999
or
p4 set -s P4CLIENT=jab.newclientname
To view this internal registry, type p4
set (with no arguments).
Debugging this...
-
If p4 help
works, then $P4PORT
is set correctly.
-
If p4 client -o
works, then $P4CLIENT
is probably set correctly; read the output closely to make sure.
-
If p4 user -o
works, then it's getting the username out of the environment (good).
-
What is/How do
I do a p4 revert?
A file can be opened for add/delete/edit/integrate/branch,
which flags the Perforce database with enough info to know who's changing
what. These "markers" are cleared when you do run
p4 submit or p4
revert on those files.
To revert a file you've opened, run
p4 revert file1 [file2 ...]
This causes several things to happen:
-
It's no longer marked as "open" in the Perforce database;
-
A fresh copy of the revision of the reverted
files that you started hacking on is copied back to your client area, since
you might've made modifications that need to be obsoleted;
-
The reverted files are marked read-only, as
if you'd just run "p4 sync" on those files.
Note that you weren't given fresh copies of the top-level
revision of the reverted files - they're left in exactly the state they
were in before you ran "p4 add/delete/edit/integrate/branch".
-
What is/How
do I do an integration? How
do I do a p4 resolve?
In short, an integration is a command to merge
changes from another codeline or a later revision of a file into a target
revision managed by Perforce.
-
For example, if you're editing revision #14 of a
file and someone checks in revision #15, you'll need to resolve
the new code into your working copy prior to submitting it to the depot.
-
Another example is if you make a change to file.c
in a branch and want to propogate the modified version to a parent/child
branch of that file.
The detailed walk-through of an integration isn't
included in this FAQ. To do this, look here
for the Perforce documentation on dealing with conflicts/integrations.
The most important thing to remember, when
you're running "p4 resolve", is that the file you're modifying is your
file; any other files are their files. Why
bother with this note? Well, much of the confusion doing the integrations
comes from questions that "p4 resolve" asked you. You're expected to respond
with things like "accept yours" or "accept theirs" or the like, and
it's easiest to remember that yours always refers to the file that
the "p4 integrate/p4 resolve/p4 submit" cycle will be modifying. It doesn't
matter whether it's the parent or the child in the branch.
[Aside: you must always remember to p4
submit the merge to complete the operation. Until that's done,
you can p4 revert the changes (to
deal with them tomorrow) or p4 resolve
-f the changes if you want a second chance at it.]
- Checkpoint/Journal questions
-
What is a checkpoint?
A checkpoint is an human-readable file that contains all information
necessary to recreate the Perforce databases but not the individual
file revision contents. It's created by running the following
command, replacing P4ROOT with the name of the Perforce server's
root directory. (The P4ROOT directory is the one that the "db.*"
files live in.)
p4d -r P4ROOT -jc
The administrator should always check the output of the command that
created the checkpoint to make sure that there were no errors.
-
When do I make a checkpoint? How often should I checkpoint my database?
This is your backup of the Perforce metadata, so it should be done
"regularly". (Probably every night or every other night is best - at
least once a week unless you've analyzed the number of changes you
make and believe that "less often" is appropriate.)
Put the checkpoint and journal files and the backup of P4ROOT/depot
on a different disk from the database!
Checkpoints are not created automatically: you must make checkpoints yourself,
using "at" (on Windows/NT) or "cron" (on Unix).
-
What's a journal file? What's it good for?
In database parlance, the journal is the running transaction log that
keeps track of all database modifications since the last checkpoint. It's the bridge between two checkpoints, and if you have Monday's
checkpoint and the journal that was collected from then until Wednesday,
those two files (as a unit) contain the same information as a checkpoint
made Wednesday would contain. This is what makes it possible to restore
the database to the most recently run transaction, instead of to
the time of the last checkpoint operation.
The journal is automatically enabled if you installed onto Windows/NT
using the InstallShield installer, but not automatically enabled on any
other platform.
If journalling is not enabled on your platform, or if you've
disabled it at some point in the past, you can enable it easily.
To enable journaling, run
p4
set P4JOURNAL=fullpathname
(on Windows/NT)
or, on Unix, set the environment variable "P4JOURNAL" to a journal
pathname prior to starting "p4d". (There's a command-line argument, "-J
journal", that will set this, also. If you use the command-line
way of setting the journal, you'll need to add that argument when you
make a checkpoint.) In either case, you'll need
to restart the Perforce server to enable this.
The instructions to do this are in the Perforce manuals, but for reference,
they are included here.
-
to restore from a checkpoint, rename the db.* files and then run:
p4d -r P4ROOT
-jr checkpoint-file
-
to restore from the journal (immediately after restoring from a checkpoint):
p4d -r P4ROOT
-jr journal-file
-
to restore from both, rename the db.* files and then run:
p4d -r P4ROOT
-jr checkpoint-file journal-file
(This operation can take several minutes to complete, depending on the
size of the checkpoint/journal files.)
Both the checkpoint-file and journal-file are made to be restored from
exactly once, that is, you cannot restore from a checkpoint file and then
try to restore again to the same "db.*" database files from that
checkpoint - you'd have to move the db.* files out of the way first.
- What do I do if the checkpoint of the database fails?
It's probably best to stop the server, and contact
Perforce Technical Support.
That said, there could be several reasons that the
checkpoint operation failed. Three possible cases follow:
-
You ran out of space on the device onto which the checkpoint data
was written. This is an easy case: clean up the space issues and try
the checkpoint operation again.
-
There was/is data corruption in the Perforce database itself. In this
case, you'll need to talk with the Perforce Support folks. If you
have the most recent successful checkpoint and the journal that was made
since then, you'll be able to restore the database easily.
-
The disk on which the db.* files live died. In this case, you'll
also want to talk to Perforce Support. The procedure to recover from
that, they'll probably tell you, is to fix the disk errors and then
restore from a checkpoint and the journal that's been kept since then.
- So if I make checkpoints and have journaling enabled, do I have a
good backup strategy?
Almost. The checkpoint+journal mechanism will protect your metadata,
but the contents of each file's revisions are stored in subdirectories
of the P4ROOT directory. Those should be backed up regularly, on the same
schedule as the checkpoint file.
-
If the machine
on which the Perforce depot dies, what do we do? If it runs out of space
on the depot filesystem, what do we do?
If the machine croaks...
If the machine croaks and you need to move the
Perforce server to another machine, you'll need to take several easy steps
(and one hard one):
-
Immediately send mail to support@perforce.com
asking for a license file for the replacement machine, unless it'll use
the same IP address as the one that died. Include the IP address
of the new machine, and be sure to stuff a $10 bill into the e-mail envelope.
-
If you can get to the disk that has the depot files,
copy them to the new machine. If not, read the section [below] about "what
if the disk croaks?".
-
Copy the new license file to the same directory as
all the "db.*" files in the depot [on the new machine].
-
Retrieve /etc/init.d/p4 from the old depot disk,
if possible, and run /etc/init.d/p4 start.
If that's not possible, set the variable P4ROOT and then run
$P4ROOT/p4d -r $P4ROOT -p 6999
This reflects some Weblogic-isms: 6999 is
our Perforce port number, and /etc/init.d/p4 is the script we use to start/stop
the server. (This script was available at Perforce.com, but....)
-
Send mail to your developers and make sure that they
change $P4PORT in their environment (to point to a new machine) prior to
using the client software.
If the disk croaks...
-
Replace the disk, and restore the database from a
backup using the checkpoint and journal files. (That's covered above,
in the previous item.)
-
Copy in a new license file, either obtained from
support@perforce.com
or from an old backup.
-
Restore the $P4ROOT/depot subdirectory from your
backups.
-
Start up the server with /etc/init.d/p4
start and stand back.
If the disk runs out of space...
-
If there wasn't database corruption, think seriously about moving $P4ROOT/depot
to another disk and symlink'ing it or mounting it back.
-
If there was, call in your local Perforce admin-types. They'll want to
restore from a checkpoint+journal combination. In this case, no data will
be lost.
-
How can
I find out who has a particular file checked out?
The command p4 opened
tells you what is opened on the current client.
p4 opened -a tells you what's opened on every
client, so
p4 opened -a | grep xxxxx
tells you every file that user xxxxx
has checked out (along with all files named xxxxx that are opened.) In
addition,
p4 opened -a filename
tells you every client that has filename opened.
-
How do
I add a new user?
-
To add a new user, just run a Perforce command as that user. It will attempt
to add them to the list of Perforce users. However...
-
... many sites use p4 protect
to turn on/off access to codelines. It's important to run this command
(as a perforce adminstrator) to grant permissions to the user.
-
What, exactly, does
p4 protect do?
-
how do
I handle security? (i.e. adding new IP address using p4 protect)
At WebLogic...
Some places use p4
protect to limit what branches are visible
to what IP addresses. For example, the main codeline is restricted to engineers
who are on the internal network or dialing in from home on dedicated IP
addresses - the output of
p4 protect
at this company is something like this:
super pambrose 209.14.136.34 //depot/dev/src030100b01/...
super rbp 209.82.17.* //depot/dev/src030100b01/...
write jab 209.82.17.* //depot/...
write jab 203.14.136.58 //depot/...
write * * -//depot/dev/src/tutorial/...
In this case, "jab" has write permission from internal IP addresses (209.82.17.*)
and from the one IP address he uses to connect to, from home (203.14.136.58).
In any case, write permission is disabled from all IP addresses ("*") for
the directory "//depot/dev/src/tutorial".
The model is that we completely restrict access to anyone not explicitly
granted access via "p4 protect". (Normally,
Perforce doesn't require this - it was a decision made based on having
people connect to Perforce from outside the internal net.)
(The above IP addresses aren't the real IP addresses used at WebLogic,
so anyone editing this FAQ should avoid "fixing" this.)
If you cannot "p4 sync" some files to the
top revision, it could be that permissions haven't been granted for that
user.
Anywhere else...
Read here
carefully. The most important details are:
-
How
do I bring an off-line worker back online
Run p4 help
diff to see what commands there are to
compare someone's local disk area to what Perforce thinks was originally
there. In particular, the following commands can be really helpful for
finding what files the developer's modified while disconnected:
cd [workspace root]
find . -type f -print | p4 -x - add
p4 diff -se ... | p4 -x - edit
p4 diff -sd ... | p4 -x - delete
Note that the "p4
-x - add" will generate errors for files that
are already in Perforce - that's okay.
[Aside: users should be told to "never run 'chmod'
or 'rm' on a file that Perforce gave you unless you're disconnected".]
-
How
do I "unsubmit" a changelist? How
do I "unsubmit" files stuck in a *pending* changelist?
If you have a file associated with pending
change #12345, you have several options:
-
Run the resolve command (or whatever the operation
was that prevented the submission from completing) and then run
p4 submit -c 12345
Revert the individual files if you want to discard
the changes, then delete the change. One such example is...
D:\weblogic>p4 describe -s 12345
Change 12345 by karena@karena.ave5 on 1998/04/16 14:44:05 *pending*
fixed links
Affected files ...
... //depot/internal/support/baystone/glossary.html#7 edit
D:\weblogic>p4 revert //depot/internal/support/baystone/glossary.html
D:\weblogic>p4 change -d 12345
(The output above is somewhat contrived, and might
not be exactly what "p4 describe"
might've produced in this situation. The important parts - the "pending"
state, the change number, and the file list, should be close enough to
use this as an example.)
Assign the files to another new change using
p4 change and p4
reopen:
D:\weblogic>p4 change
(lots of edits of the change description, saving
the output in the editor. Let's assume that the change number that it created
was change #34346.)
D:\weblogic>p4 reopen -c 34343 //depot/internal/support/baystone/glossary.html
Of course, you'll still want to delete that pending change at some point.
-
Should
I back up clients? If so, what files should I back up and what files should
I keep? How long should I keep backups of clients?
This is specific to each company that implements
Perforce.
Back up the depot itself...
The most important thing to back up is the Perforce
depot, which is a set of three things: the checkpointed database,
the subdirectory (called "depot") that stores each revision of each file
that Perforce is managing, and the license file.
If your Perforce Adminstrators are on the ball...
The information to recreate products should be
stored as files in Perforce, and the release should be recreateable
using only the files in Perforce. The "label" information is stored in
the Perforce databases (and hence, is in the backed-up checkpoint file),
so releases should be safe.
But should I back up a Perforce client?
-
Strictly speaking, it should not be necessary if
developers check in work frequently.
-
That said, it probably makes sense to archive the
Perforce client area that generates a release that's sent to customers,
so that when a patch needs to be made it can be built from exactly the
same source and object files that created the original release. (You
don't want to have to recompile everything to send out a patch,
unless you're regressing everything in sight. Compilers might've changed,
machine environments might've changed. Why borrow trouble?)
So I don't need to back up any Perforce clients
except my build clients?
Talk to your local Perforce Administrator-types.
They'll have an opinion on this, we can assure you.
Last modified on 21 September 1998.