What is the Server Deployment Package (SDP?) === Overview --- The Server Deployment Package (SDP) provides a set of management scripts for operating various Perforce software products, such as: * p4d * p4broker * p4p * p4web * p4dtg * p4ftpd For **p4d** in particular, the SDP provides a sample implementation of best practices, such as performing _offline checkpoints_, rotating logs and journals routinely, and doing a database sanity/safety check before starting the server. The SDP provides a structure for operating multiple instances of Perforce on a given machine, allowing for the possibility of each instance having different configuration aspects, e.g. * versions of software components (p4d, p4broker, etc.) * case sensitivity setting (sensitive/insensitive) * Unicode setting (enabled/disabled) Bugs, Enhancements, and Future Plans === Perforce Jobs are used to file bugs and request enhancements to the SDP. The list of jobs (other than closed ones) can be viewed [here](https://swarm.workshop.perforce.com/projects/perforce-software-sdp/jobs "SDP Open Jobs"). You can see all jobs (including closed ones if desired) from the command line interface. Connect to The Workshop and run the p4 jobs command, something like this: p4 jobs -l -e "Project=perforce-software-sdp ^status=closed" There is no formal roadmap for the SDP. Contributing === Contributions to the SDP are most welcome! Note that contributing requires a basic user knowledge of working with Perforce. Contributing by Shelving --- All registered Workshop users have **open** access (but not **write**) to the SDP, specifically to this path: //guest/perforce_software/sdp/main/... The **open** access level confers the ability to edit and **shelve** changes for a pre-commit review process. We'll review the change, and either incorporate it or provide feedback. Be sure to provide a detailed change description, and also include the tag/text "[review]" in your changelist description to initiate a Swarm code review. Contributing by Branching --- Optionally, you may also branch the SDP folder, or some subset of it, into your own guest area. Creating a branch spec is recommended for this purpose, e.g. with a branch spec like this: Branch: your_name-sdp Owner: your_name Description: SDP Updates by your_name. Options: unlocked View: //guest/perforce_software/sdp/main/Server/Unix/etc/... //guest/your_name/sdp/main/Server/Unix/etc/... If you branch a subset, it's a good idea to maintain the full path structure (as in the example above), so you can easily expand the scope of what you branched initially at some later date. Edit, test and submit changes in your branch. Then when you are done, merge your changes back into our mainline, shelving them for review. Guidelines for SDP Contributors --- * Communicate using Swarm! Let folks know what you're planning before you work to hard on something that might already be underway. * File a job (Project=**perforce-software-sdp**) first. That's not required, but increases the transparency of what you're doing, and helps promote it. * We prefer changes that are generic, and have a wide appeal. * The core of the SDP, the scripts that do offline checkpoints for p4d, prioritize Perforce best practices over scripting magic. Don't get too fancy! :) Versioning the SDP Locally === In line with the "Version Everything" mindset, the SDP itself should be versioned in your environment. Commonly the SDP is versioned in a depot such as //Perforce or //p4admin, with a structure like this: //p4admin/p4c/sdp/... //p4admin/main/sdp/... //p4admin/host//p4/1/bin/... The **p4c** and **main** folders are branches. The **p4c** branch is designated for receiving generic SDP updates from The Workshop. The **main** branch is where local customizations and/or configuration changes are applied. The **host** folder contains instance-specific and host-specific files, which effectively communicate part of your topology. These define which services are active on which hosts. On the Perforce server machines, basic workspaces are used to manage the SDP. Recent versions of the SDP contain a .p4ignore file that helps avoid versioning of files that do not belong in version control, things like password files, editor temp files, and generated/compiled Python cache files. Perforce Swarm and Perforce Git Fusion --- When installing Perforce Swarm or Perforce Git Fusion in the SDP structure, they can and should be deployed in a way so as to benefit from the SDP structure, e.g. instances of Swarm and/or Git Fusion to run against each p4d instance. Details for doing this are not yet incorporated the mainline SDP, though they have been deployed at customer sites. However, the basic ideas are fairly straightforward. As a hint for Swarm, you'll want to apply the SWARM_DATA_PATH in the httpd configuration to reference Swarm for each SDP instance. Upgrading Existing SDP Installations === If you have an earlier version of the Server Deployment Package (SDP) installed, you'll want to be aware of the new -test flag to the SDP setup script, mkdirs.sh. The following update instructions assume a simple, single-server topology. Get the new software from The Workshop, which you'll find in the form of a SDP.Unix.*.tgz zipped tar file. Put the tar file in the /depotdata folder on your server. **BE SURE TO MOVE THE OLD** /depotdata/sdp folder aside first if it is there, e.g. to /depotdata/sdp.old.2014-10-08. Then unpack it like this: tar -xzvpf SDP.Unix.2014.2.####.tgz cd /depotdata/sdp/Unix/setup Modify the mkdirs.sh script to suit your environment (possibly borrowing values from your old mkdirs.sh script). Then run it: mkdirs.sh 1 -test That'll do a mock install, creating a **/tmp/**p4 structure, with folders like /tmp/p4/common/bin and /tmp/p4/1/bin. The output will suggest some diff -r commands to run to to view the diffs and apply updates manually, being careful not to lose any customizations or local configuration changes you may have made. If the SDP itself is versioned in Perforce in your environment, you can also put the new things from /tmp/p4 into your Perforce server where the SDP is. Commonly, there is a branch for receiving pristine updates of the generic SDP, e.g. 'p4c', which allow you to use Perforce merging to merge update into your local SDP mainline. Contact [Perforce Consulting Services](mailto:consulting@perforce.com) to ask about engaging professional services to upgrade the SDP for you. A few quick things to be aware of: * The old p4_1_init script (more generically p4_instance_init) was renamed to p4**d**_1_init. This will require updates to init script or symlink, e.g. in /etc/init.d. * Once you upgrade p4d to 2014.1+, you can do away with the SDP trigger that updates depot spec map fields. You can clear the Map: fields of depots, and use the 2014.1 server.depot.root configurable. Going forward, you'd only use Map: fields for depots that really do require a unique path, e.g. archive depots. SDP Structure === The SDP structure enables two major views of a Perforce server, the system administrator/storage architect view, and the application administrator view. THe system admin sees this structure: /depotdata /metadata /logs System administrators know they need to backup /depotdata. Backing up /logs is optional, and /metadata is to be avoided by backup utilities, virus scanners, and anything else that might touch files in that area. This view is designed for optimizing for performance, as each of those folders has different performance profiles and usage patterns demanded by p4d. This view is not concerned with how many instances there are, or how they vary. Application administrators (Perforce Admins), though a structure of symlinks, see Perforce from a view like this: /p4/1/bin /p4/1/config /p4/1/depots /p4/1/root /p4/1/ssl /p4/1/tmp /p4/common/bin This structure illustrates an instanced named '1' (it need not be numeric), as a /p4/common area which contains files common to all instances. Each of those folders is physically stored under an appropriate storage folder or volume as defined in the system admin view above. The /p4/_instance_/bin folder defines which Perforce products are active for that instance on the current machine (p4d, p4broker, p4p, etc.). The **root** folder contains P4ROOT, while **depots** contains archive files. THe SDP provides standard mechanism for defining a controlled shell environment. This benefits p4d and other Perforce software products directly, and also provides good examples of controlling the environment for supporting automation, such as Perforce standards guidance and policy enforcement scripts (e.g. p4d triggers or p4broker filter scripts). For advanced deployments, the SDP defines a standard for installing and configuring derived APIs, such as P4Python into /p4/common/python. Many Perforce customers use the SDP as-is. Others benefit by incorporating concepts and ideas from the SDP into their own environments. The Windows SDP === There are two flavors of the SDP, Unix and Windows. Both are available in the same folder structure in the Workshop. Testing the SDP === There is automated tests using Vagrant for the Unix variant of the SDP. See test/README.md for details.