Perforce FAQ for System Administrators

Contributed by Jeff Bowles

Note - the intended audience for this FAQ is a system administrator who knows some Perforce commands as a user, but needs more information to be able to do some Perforce administration tasks such as setting up a new Perforce user or dealing with restarting the Perforce server after a recovery from a catastrophic disk crash.

This FAQ assumes that there is a Perforce Administrator who knows some of the more complex parts of Perforce, who can provide help, counseling, and deal with policy-specific issues....

What's a client? What's a client spec? How do I tell what this client spec does and doesn't do? How do I add a new client?

Glossary

depot - the term usually applied to the Perforce repository, which lives on the other end of the network connection specified with $P4PORT. There's also a syntax used to specify the path names of files in the depot: //depot/x/y/z/file.java.
client - the [local] disk area on a developer's machine, onto which read-only copies of the source files are copied. Usually there's one client per user and one client per machine, but that's completely up to the user and to the people who set it up. (The command p4 client allows you to change the mapping so that a file is stored, locally, to the directory/file you specify. See the next item for details.)
client specification - documented here, the internal information specific to a Perforce client, that specifies the following:

The date/time the client was created.
The local directory name that's the top of the local copy of the file set retrieved from the depot.

Root:   D:/src

An owner, which isn't that useful.
Lines at the end that look like

//depot/x/y/...    //clientname/x/y/...
//depot/x/z/%1     //clientname/x/old-z/%1

The way to read this is "left is the depot pathname and right is the local disk pathname". These lines are the meat of the client specification and can be a little tricky.

The first line says "all files in depot directory /x/y on the depot should be installed in D:/src/x/y, preserving the path names below x/y" (for this example). Every "..." is a wild card meaning "everything from here down".
the second lines says "the files in depot directory /x/z (but no subdirectories in it) should be put into D:/src/x/old-z".

The default client spec maps all depot files to the local disk, using:

//depot/...    //clientname/...

Usually we'll make a client with the "default client specification" and use that to create new clients. For example, if the a client named XXX exists and has the perfect client spec for a new client to imitate, then

P4CLIENT=newclientname
p4 client -t XXX

will create the new client using the existing "XXX" client as a template. (Hint:

p4 client

uses the current directory as the default for the Root: of the local file set it'll retrieve, so you can save keystrokes by

chdir

'ing there before running

p4 client

client software - the copy of p4.exe or p4 ("client") that someone's downloaded from Perforce, Inc. that connects to a database process ("server") that administers the depot. There are many platforms supported, which means there are many different machines for which you can download a new p4 client executable. [Aside: there are no legal restrictions at all w.r.t. Perforce client software - the licenses are based on the number of legal users, and we can download client software for as many different platform types as we choose.]

How do I tell what this client spec does and doesn't do?

p4 client -o clientname will tell you a lot about a remote client. The two items to stare at are "Root:" (where the local files are loaded) and "View:" (the mapping from depot-name to local-name).
Most client-spec headaches have to do with the view not including a new branch/codeline recently installed - your build engineers can usually alert you these problems - and problems with wild cards in a "View:" line. ("..." and "%1", above, are both such wild cards and are documented here. In general, your build engineers and/or your local Perforce administrators should have a handle on this.)

How do I add a new client?

See the item, above, that talks about p4 client -t XXX.

How do you add/delete files? Can you retrieve a deleted file? How?

To add a file, run p4 add filename; to delete a file, run p4 delete filename.

p4 submit

To retrieve a deleted file:

You need to know the filename in the form:

//depot/dev/src/sandbox/stella/GNUdepends.def

You'll need some history information, so run the following command:

D:\src\dev\src>p4 filelog //depot/dev/src/sandbox/stella/GNUdepends.def
//depot/dev/src/sandbox/stella/GNUdepends.def
... #4 change 19737 delete on 1998/05/05 by stella@stella.greenwich 'Change to move stella's erro'
... ... delete into //depot/dev/src030100b01/sandbox/stella/GNUdepends.def#2
... #3 change 19611 edit on 1998/05/04 by stella@stella.sol 'Bits of hacking to support stag'
... ... branch into //depot/dev/sandbox/stella/GNUdepends.def#1
... ... branch into //depot/dev/src030100b01/sandbox/stella/GNUdepends.def#1

In this case, revision #4 deleted the file, and revision #3 was the last revision that had "real" contents.

A fast way to look at the contents of revision #3 of this file:

p4 print  //depot/dev/src/sandbox/stella/GNUdepends.def#3

Note that the first line of output is a line of the form "

//depot/dev/src/sandbox/stella/GNUdepends.def#3

If you don't want that, run

p4 print -q

instead of

p4 print

To retrieve the contents and check 'em in again, you need to "re-add" the file:

p4 print -q  GNUdepends.def#3 > newfile
p4 add GNUdepends.def
cp newfile  GNUdepends.def
p4 submit

(There's two other ways to do this: one is to branch revision #3 of the GNUdepends.def, creating a new version, but unfortunately it would need a different name; another would be to "p4 sync -f GNUdepends.def#3" and then move that copy to the side, then do the "p4 add" shown above.)

The unasked question is Can I expunge a file that I checked into the wrong place? That question is answered in the Perforce documentation, under "p4 obliterate", but this shouldn't be done without your Perforce administrator standing over you. (That operation is completely irreversible.)

How does Perforce deal with CR/LF issues? Can this be overridden?

Background

for text files

How do you tell if a file is "text" or not?

If a file's checked in, run p4 file XXX where XXX is the name of the file. You'll get output of the form:

//depot/dev/src/GNUdepends.def#3 - edit change 20027 (ktext)
//depot/dev/src/GNUmakefile#188 - edit change 21753 (text)
//depot/native/events/EventsGenCtl.bmp#3 - delete change 1103 (binary)

Unless it was specified when the user ran p4 add, the local Perforce client guessed based on the first few bytes of the file. (NUL characters and non-ASCII are dead giveaways - if they're present, the file has to be binary.)

What if you it guessed wrong, and you need to change the type of a file so that it does end-of-line processing?

You'll need to open the existing file for editing, change its type, and submit the change:

p4   edit  -t text  //depot/native/events/EventsGenCtl.bmp
p4   submit

In this case, you'll probably be treating a .bmp file like it's text, which is pretty silly, but this gets the point across. You can go back and forth with a file, if you choose - change the type of a file from text to binary, or binary to text. (The

ktext

p4 help reopen

Ways to force it to not interpret end-of-line, to leave the file contents alone:

How do I get Unix end-of-line processing behavior on my Windows Perforce client?

Short answer: I don't know. Certainly, you can edit the files on Unix, store what you want there, and change the file type to "binary" to guarantee that the contents aren't munged when you fetch 'em with the Windows client. that's probably the fastest, but ugliest way.

How do I tell what some else's client spec looks like? How do I change the e-mail address for a user is, if it's not the same as their P4 user name?

Looking at someone else's client...

To look at your client, you run:

p4 client

(dumps you into an editor)

p4 client -o

(dumps information to standard output)

To look at someone else's client, run:

p4 -c client-name client

p4 -c client-name  client -o

(Don't change this information (inside the editor) without eventually informing the user who uses it the most - they could get cranky about it!)

Looking at someone else's user information...

To look at your user information, you run:

p4 user

(dumps you into an editor)

p4 user -o

(dumps information to standard output)

To look at someone else's user information, you run:

p4 user

their-name

(dumps you into an editor)

p4 -u their-name user -o

(dumps information to standard output)

(Note that, if it dumped you into an editor, the file's readonly. To modify another user's information, you will need to utter a couple of extra commands not included in this FAQ - see your local Perforce Administrator for those. )

How would I change someone's e-mail address in the Perforce user database?

Run

p4 user user-name

(which dumps you into an editor) and change it.

Note - if you're trying to automate this...

Most commands that use the option

-o

to write to standard output have a

-i

option to read from standard input. By doing this, you could write a script that appends "@myplace.com" to the e-mail user with the commands:

p4 user -o > tmp-file
sed '/^Email:/s/$/@myplace.com/' < tmp-file | p4 user -i

The underlined part will need a modification to succeed - it's deliberately omitted from this FAQ for security reasons. See your Perforce Adminstrators for details.

(If you do this sorta thing, put a lot more error checking into the script that you see here!)

Should we store binaries in Perforce? Why, or why not?

Sure, but they can take up some space. There are several things you need to do when you store a binary in Perforce:

Make sure its file type is "binary" (or "xbinary") when you check it in. Binary files are stored in a different manner than text files, because with text files it makes sense to store a simple delta between two revisions. For certain generated text files (PostScript, Adobe Acrobat PDF files) and binaries it doesn't make sense, so pick the file type appropriately.
Perforce stores enough information to recreate each revision of the binary, so if the binary is 100 Mb and you keep checking in new versions every day, you might run out of disk very quickly.
A binary file stored in a branch doesn't immediately take up more space, but has the potential to fill up each user's client area by having multiple copies of [potentially] big files. [This can happen because a user might have multiple branches mapped to their disk, and they'd get a separate copy of the file for each branch.]

The main headache with storing binaries is that it's tempting to check in the complete release you developed using Perforce, so that "x.c" and "x.h" and "x.o" and "x.out" all appear in Perforce. That might not be the strategy you want, partly because it introduces an ambiguity: someone might update "x.c" or "x.h" but not realize that "x.out" is getting shipped, but wasn't updated. (That's dangerous.)

Addendum: a recent version of Perforce supports "compressed files" and can save a lot of space on the depot when used. If you need to store binaries in Perforce, it's handy to research this new feature.

Also, there's a new file type called "tempobj" which stores "temporary object" files. It's a good way to check in the most recent revision of some object file, obsoleting all prior revisions, so that developers will be able to use this object (or library or executable) without recompiling it.

On Windows/NT, how can I look at internal variables that the Perforce client uses to connect?

Environment

There are the environment variables described here, but the most frequently used are $P4PORT and $P4CLIENT.

Internal variables

In addition, p4 set shows some entries hidden [in the Windows registry, I believe] that can be set using

p4 set  P4PORT=p4server:1666

p4 set -s P4CLIENT=stella.newclientname

To view this internal registry, type

p4 set

(with no arguments).

Debugging this...

If p4 help works, then $P4PORT is set correctly.
If p4 client -o works, then $P4CLIENT is probably set correctly; read the output closely to make sure.
If p4 user -o works, then it's getting the username out of the environment (good).

What is/How do I do a p4 revert?

A file can be opened for add/delete/edit/integrate/branch, which flags the Perforce database with enough info to know who's changing what. These "markers" are cleared when you do run

p4 submit

p4 revert

on those files.

To revert a file you've opened, run

p4 revert  file1   [file2  ...]

This causes several things to happen:

It's no longer marked as "open" in the Perforce database;
A fresh copy of the revision of the reverted files that you started hacking on is copied back to your client area, since you might've made modifications that need to be obsoleted;
The reverted files are marked read-only, as if you'd just run "p4 sync" on those files.

Note that you weren't given fresh copies of the top-level revision of the reverted files - they're left in exactly the state they were in before you ran "p4 add/delete/edit/integrate/branch".

What is/How do I do an integration? How do I do a p4 resolve?

In short, an integration is a command to merge changes from another codeline or a later revision of a file into a target revision managed by Perforce.

For example, if you're editing revision #14 of a file and someone checks in revision #15, you'll need to resolve the new code into your working copy prior to submitting it to the depot.
Another example is if you make a change to file.c in a branch and want to propogate the modified version to a parent/child branch of that file.

The detailed walk-through of an integration isn't included in this FAQ. To do this, look here for the Perforce documentation on dealing with conflicts/integrations.

The most important thing to remember, when you're running "p4 resolve", is that the file you're modifying is your file; any other files are their files. Why bother with this note? Well, much of the confusion doing the integrations comes from questions that "p4 resolve" asked you. You're expected to respond with things like "accept yours" or "accept theirs" or the like, and it's easiest to remember that yours always refers to the file that the "p4 integrate/p4 resolve/p4 submit" cycle will be modifying. It doesn't matter whether it's the parent or the child in the branch.

p4 submit

p4 revert

tomorrow

p4 resolve -f

Checkpoint/Journal questions
1. What is a checkpoint?
  A checkpoint is an human-readable file that contains all information necessary to recreate the Perforce databases but not the individual file revision contents. It's created by running the following command, replacing P4ROOT with the name of the Perforce server's root directory. (The P4ROOT directory is the one that the "db.*" files live in.)
```
 p4  admin  checkpoint
```
  In older versions of Perforce that don't have the p4 admin subcommand:
```
 p4d   -r   P4ROOT   -jc
```
  The administrator should always check the output of the command that created the checkpoint to make sure that there were no errors.
2. When do I make a checkpoint? How often should I checkpoint my database?
  This is your backup of the Perforce metadata, so it should be done "regularly". (Probably every night or every other night is best - at least once a week unless you've analyzed the number of changes you make and believe that "less often" is appropriate.)
  
  Put the checkpoint and journal files and the backup of P4ROOT/depot on a different disk from the database!
  
  Checkpoints are not created automatically: you must make checkpoints yourself, using "at" (on Windows/NT) or "cron" (on Unix).
3. What's a journal file? What's it good for?
  In database parlance, the journal is the running transaction log that keeps track of all database modifications since the last checkpoint. It's the bridge between two checkpoints, and if you have Monday's checkpoint and the journal that was collected from then until Wednesday, those two files (as a unit) contain the same information as a checkpoint made Wednesday would contain. This is what makes it possible to restore the database to the most recently run transaction, instead of to the time of the last checkpoint operation.
  The journal is automatically enabled if you installed onto Windows/NT using the InstallShield installer, but not automatically enabled on any other platform.
  If journalling is not enabled on your platform, or if you've disabled it at some point in the past, you can enable it easily. To enable journaling, run
  p4 set P4JOURNAL=fullpathname (on Windows/NT)
  or, on Unix, set the environment variable "P4JOURNAL" to a journal pathname prior to starting "p4d". (There's a command-line argument, "-J journal", that will set this, also. If you use the command-line way of setting the journal, you'll need to add that argument when you make a checkpoint.) In either case, you'll need to restart the Perforce server to enable this.
  The instructions to do this are in the Perforce manuals, but for reference, they are included here.
  - to restore from a checkpoint, rename the db.* files and then run:
  - to restore from the journal (immediately after restoring from a checkpoint):
  - to restore from both, rename the db.* files and then run:
  (This operation can take several minutes to complete, depending on the size of the checkpoint/journal files.)
  Both the checkpoint-file and journal-file are made to be restored from exactly once, that is, you cannot restore from a checkpoint file and then try to restore again to the same "db.*" database files from that checkpoint - you'd have to move the db.* files out of the way first.
4. What do I do if the checkpoint of the database fails?
  It's probably best to stop the server, and contact Perforce Technical Support.
  That said, there could be several reasons that the checkpoint operation failed. Three possible cases follow:
  1. You ran out of space on the device onto which the checkpoint data was written. This is an easy case: clean up the space issues and try the checkpoint operation again.
  2. There was/is data corruption in the Perforce database itself. In this case, you'll need to talk with the Perforce Support folks. If you have the most recent successful checkpoint and the journal that was made since then, you'll be able to restore the database easily.
  3. The disk on which the db.* files live died. In this case, you'll also want to talk to Perforce Support. The procedure to recover from that, they'll probably tell you, is to fix the disk errors and then restore from a checkpoint and the journal that's been kept since then.
5. So if I make checkpoints and have journaling enabled, do I have a good backup strategy?
  Almost. The checkpoint+journal mechanism will protect your metadata, but the contents of each file's revisions are stored in subdirectories of the P4ROOT directory. Those should be backed up regularly, on the same schedule as the checkpoint file.

If the machine on which the Perforce depot dies, what do we do? If it runs out of space on the depot filesystem, what do we do?

If the machine croaks...

If the machine croaks and you need to move the Perforce server to another machine, you'll need to take several easy steps (and one hard one):

Immediately send mail to support@perforce.com asking for a license file for the replacement machine, unless it'll use the same IP address as the one that died. Include the IP address of the new machine, and be sure to stuff a $10 bill into the e-mail envelope.
If you can get to the disk that has the depot files, copy them to the new machine. If not, read the section [below] about "what if the disk croaks?".
Copy the new license file to the same directory as all the "db.*" files in the depot [on the new machine].
Retrieve /etc/init.d/p4 from the old depot disk, if possible, and run /etc/init.d/p4 start. If that's not possible, set the variable P4ROOT and then run

$P4ROOT/p4d -r $P4ROOT -p 1666

Your site might use some other port number, like maybe 6999, and that'll go after the `-p' parameter.

Send mail to your developers and make sure that they change $P4PORT in their environment (to point to a new machine) prior to using the client software.

If the disk croaks...

Replace the disk, and restore the database from a backup using the checkpoint and journal files. (That's covered above, in the previous item.)
Copy in a new license file, either obtained from support@perforce.com or from an old backup.
Restore the $P4ROOT/depot subdirectory from your backups.
Start up the server with /etc/init.d/p4 start and stand back.

If the disk runs out of space...

If there wasn't database corruption, think seriously about moving $P4ROOT/depot to another disk and symlink'ing it or mounting it back.
If there was, call in your local Perforce admin-types. They'll want to restore from a checkpoint+journal combination. In this case, no data will be lost.

How can I find out who has a particular file checked out?

The command

p4 opened

tells you what is opened on the current client.

p4 opened -a

tells you what's opened on every client, so

p4 opened -a | grep sitemap

tells you every file that user sitemap has checked out (along with all files named sitemap that are opened.) In addition,

p4 opened -a   filename

tells you every client that has filename opened.

How do I add a new user?

To add a new user, just run a Perforce command as that user. It will attempt to add them to the list of Perforce users. However...
... many sites use p4 protect to turn on/off access to codelines. It's important to run this command (as a perforce adminstrator) to grant permissions to the user.

What, exactly, does p4 protect do?

p4 protect

d:\src\dev> p4 protect

# comments deleted

Protections:

super user root 207.82.17.* //depot/...

super user maggie 207.82.17.* //depot/...

super user maggie 206.14.136.34 //depot/...

super user brick 207.82.17.* //depot/...

super user brick 206.14.136.65 //depot/...

super user brick 209.185.70.* //depot/...

super user kevin 207.82.17.* //depot/...

super user stella 206.14.136.58 //depot/...

write user laurie 207.82.17.* //depot/...

write user carl 207.82.17.* //depot/dev/src/...

read user carl 206.14.136.44 //depot/dev/src030100b01/...

write user bigdaddy 207.82.17.* //depot/...

The way to read this is:

permission user username IP-Address depot-pathname

This means "this username, at this IP-Address, has the given permissions on files that match this "depot" pathname." So, for example:

User "root" can do protected operations on the entire depot, but only from the internal company IP addresses. (Line 1)
User "maggie" can do protected operations from the internal net and from his home IP address. (Lines 2-3)
Carl can modify things in //depot/dev/src but can only read files in /depot/dev/src030100b01, and in that case, only from home.

By default, if the permission isn't explicitly granted in the "p4 protect" spec, it's denied completely to that user.

Note that this doesn't cover the mechanism added, in release 98.2, for "groups". That's covered in the manual.<

how do I handle security? (i.e. adding new IP address using p4 protect)

At WebLogic...

Some places use

p4 protect

to limit what branches are visible to what IP addresses. For example, the main codeline is restricted to engineers who are on the internal network or dialing in from home on dedicated IP addresses - the output of

p4 protect

at this company is something like this:

super maggie 209.14.136.34 //depot/dev/src030100b01/...

super brick 209.82.17.* //depot/dev/src030100b01/...

write stella 209.82.17.* //depot/...

write stella 203.14.136.58 //depot/...

write * * -//depot/dev/src/tutorial/...

The model is that we completely restrict access to anyone not explicitly granted access via "p4 protect". (Normally, Perforce doesn't require this - it was a decision made based on having people connect to Perforce from outside the internal net.)

(The above IP addresses aren't the real IP addresses used at WebLogic, so anyone editing this FAQ should avoid "fixing" this.)

If you cannot "p4 sync" some files to the top revision, it could be that permissions haven't been granted for that user.

Anywhere else...

here

There's no easy way to include comments in the "p4 protect" maps. We've used the following syntax to include comments:

write * * -//depot/comment/***
write * * -//depot/comment/***_1a_remove_the_permissions_for_frozen_branches_here
write * * -//depot/comment/***

Order in the p4 protect specification is significant. The first permissions granted can be overridden by later specs, so "less is more".

How do I bring an off-line worker back online

Run

p4 help diff

to see what commands there are to compare someone's local disk area to what Perforce thinks was originally there. In particular, the following commands can be really helpful for finding what files the developer's modified while disconnected:

cd [workspace root]
find . -type f -print | p4 -x - add
p4 diff -se ... | p4 -x - edit
p4 diff -sd ... | p4 -x - delete

Note that the "p4 -x - add" will generate errors for files that are already in Perforce - that's okay.

[Aside: users should be told to "never run 'chmod' or 'rm' on a file that Perforce gave you unless you're disconnected".]

How do I "unsubmit" a changelist? How do I "unsubmit" files stuck in a *pending* changelist?

If you have a file associated with pending change #12345, you have several options:

Run the resolve command (or whatever the operation was that prevented the submission from completing) and then run

p4 submit -c 12345

Revert the individual files if you want to discard the changes, then delete the change. One such example is...

D:\src>p4 describe -s 12345

Change 12345 by karena@karena.ave5 on 1998/04/16 14:44:05 *pending*

fixed links

Affected files ...

... //depot/internal/support/baystone/glossary.html#7 edit

D:\src>p4 revert  //depot/internal/support/baystone/glossary.html

D:\src>p4 change -d 12345

(The output above is somewhat contrived, and might not be exactly what "

p4 describe

" might've produced in this situation. The important parts - the "pending" state, the change number, and the file list, should be close enough to use this as an example.)

Assign the files to another new change using p4 change and p4 reopen:

D:\src>p4 change

(lots of edits of the change description, saving the output in the editor. Let's assume that the change number that it created was change #34346.)

D:\src>p4 reopen -c 34343  //depot/internal/support/baystone/glossary.html

Of course, you'll still want to delete that pending change at some point.

Should I back up clients? If so, what files should I back up and what files should I keep? How long should I keep backups of clients?

This is specific to each company that implements Perforce.

Back up the depot itself...

The most important thing to back up is the Perforce depot, which is a set of three things: the checkpointed database, the subdirectory (called "depot") that stores each revision of each file that Perforce is managing, and the license file.

If your Perforce Adminstrators are on the ball...

The information to recreate products should be stored as files in Perforce, and the release should be recreateable using only the files in Perforce. The "label" information is stored in the Perforce databases (and hence, is in the backed-up checkpoint file), so releases should be safe.

But should I back up a Perforce client?

Strictly speaking, it should not be necessary if developers check in work frequently.
That said, it probably makes sense to archive the Perforce client area that generates a release that's sent to customers, so that when a patch needs to be made it can be built from exactly the same source and object files that created the original release. (You don't want to have to recompile everything to send out a patch, unless you're regressing everything in sight. Compilers might've changed, machine environments might've changed. Why borrow trouble?)

So I don't need to back up any Perforce clients except my build clients?

Talk to your local Perforce Administrator-types. They'll have an opinion on this, we can assure you.

Perforce FAQ for System Administrators

Client Stuff

Non-admin Usage Stuff

Policy Stuff

Admin Issues (backups et al)