# Install SDP Test Server Setup

## SAFETY WARNING

**Use this procedure ONLY on an expendable virtual machine on which there is NO DATA OF ANY VALUE.**

The new SDP `install_sdp.sh` that this procedures tests is very safe.  However, the test suite for it is rather dangerous, especially the script clearly named `DANGER_CLEAN.sh`. This is called automatically, several times, when the full test suite is executed.

## Overview

These are instructions for setting up a test server for testing the `install_sdp.sh` script.  This procedure is intended to allow testing of pre-release and dev branch versions of the script and related software in the SDP, such as the `mkdirs.sh` script, on various Linux distributions and UNIX operating systems.

This procedure is different (and slightly more complex) than what a customer would do. A customer will use a simpler procedure that grabs only the `install_sdp.sh` with a curl command, and then that script in turns does a curl to grab the SDP tarball. This procedure pulls the latest files from the SDP dev branch, and thus has extra steps.  This procedure also pulls additional files that the real install process will be able to pull dynamically, such as P4 binaries and the Sample Depot tarball.

## Platforms

Installs have been done on the following platforms thus far:

* Ubuntu 26.04 - NOT APPROVED FOR PRODUCTION USE until [this Ubuntu bug](https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2161574) is fixed; that bug contains a workaround for testing.
* Ubuntu 24.04
* Ubuntu 22.04
* Ubuntu 20.04 - Works but no longer supported due to this platform having gone EOL for standard support on June 30, 2026. ESM still available until 2030.
* Rocky 10
* Rocky 9
* Rocky 8
* SLES 15

It is expected to work without modification on:

* RHEL 8
* RHEL 9
* RHEL 10

Currently there are issues on these platforms:

* Ubuntu 26.04 - Issues with breaking of POSIX compliance for the 'chown' command cause issues with SDP installation. For more information, see [this Ubuntu bug](https://bugs.launchpad.net/ubuntu/+source/rust-coreutils/+bug/2161574) is fixed; that bug contains a workaround for testing.

Please update this page or contact p4consulting@perforce.com to indicate successful operation on additional platforms and/or issues getting it to work on other platforms.

## Procedure for Installing and Operating SDP install_sdp.sh Test Suite.

**STEP 1**: Update your machine image to latest OS patches.

Use the mechanism appropriate for your platform, which should look like one of these (executed as `root`):

On Rocky/Red Hat systems:

```
sudo dnf update -y
```

Use `yum` if `dnf` isn't available (e.g. on older systems dervied from Red Hat, such as such as CentOS).

Ubuntu:

```
sudo apt update 
sudo apt upgrade -y 
sudo apt autoremove -y
```

On SuSE systems:

```
sudo zypper update -y
```

After this general update of OS packages is done, do a reboot. On all platforms, this is done like so, as root:

```
sudo reboot
```

_Optional_:  If you find it convenient to install a preferred text editor, do so at this time. For example, on Ubuntu systems, you might do:

```
sudo apt install -y neovim
```

Replace `neovim` in the above example with the package name for your preferred text editor.

Also, install the ShellCheck package on the machine if you plan to do development work. That is critical for working on bash shells scripts, but is not required to execute the test suite.

**STEP 2**: Add 'dev' alias to ~/.bashrc of sudo user.

NOTE: If you login to your your test server directly as root, skip this step.

Add 'dev' alias to the ~/.bashrc of the user with sudo that you login to the machine with, e.g. ubuntu, rocky, ec2-user, etc.

Append this one line to the bottom of the ~/.bashrc file:

```
alias dev='sudo su -'
```

**STEP 3**: Bootstrap the Machine

Everything from here through having a fully working shell environment -- acquiring P4 binaries, cloning both the test suite and the SDP, installing required OS packages, and setting up the `dev` alias and prompt -- is handled by `bootstrap_test_machine.sh`.

Since this machine has no P4 client tooling yet, pull the script itself via `curl` from P4 Code Review's raw-download URL (see the "P4 Code Review (fka Helix Swarm) URLs" section in `ai_dev_support/P4Info.md` for background on that URL pattern), then run it.

Become root first if you're not already (`sudo su -`), then:

```
curl -O https://workshop.perforce.com/download/test-install_sdp/dev/bootstrap_test_machine.sh
chmod +x bootstrap_test_machine.sh
./bootstrap_test_machine.sh -mode basic
```

Use `-mode advanced` instead of `-mode basic` if you want read/write access (requires an account on the Public Depot server -- to request one, contact p4consulting@perforce.com -- and logging in first: `p4 -u <youraccount> -p public.perforce.com:1666 login`). Any required option not given on the command line -- including `-mode` itself -- is prompted for interactively.

By default this pulls the test suite from `//test-install_sdp/dev` and the SDP from `//p4-sdp/dev_rebrand` (where SDP development is happening as of this writing), bootstraps with the current P4D release, and auto-detects this machine's `PlatformTag` (e.g. `r9x86_64`) from `/etc/os-release` and `uname -m`. All of that can be overridden -- see `./bootstrap_test_machine.sh -man` for the full option list, including the `PlatformTag` auto-detection mapping table and how to extend it for a new platform.

Ensure the output ends with `Success: bootstrap_test_machine.sh completed with no errors or warnings.`  (Some warnings, e.g. a failure to set the hostname, are non-fatal and safe to ignore; errors are not.)

**STEP 4**: Reset environment.

Now, exit entirely out of the machine, and then ssh back in as the user with sudo prives that you login to the machine with.  Use the two `dev` aliases defined earlier -- one for the sudo user, and the next one for the root user.  If you skipped **STEP 2** (because you login directly as root and don't have a sudo user), call the `dev` alias only once.

Do like so:

```
dev             # <-- As the sudo user, to become root.
dev             # <-- As the root user, to setup the shell environment.
cdd             # <-- To change to the directory where the test suite scripts r
r ls            # <-- To see test groups in the test suite.
```

That last command will display a list of test groups something like this:

```
Usage
PreflightAbort
DryRun
OnlineInstallBasic
OnlineInstallBasicOldP4D
OnlineInstallWithSD
LocalInstallEmpty
LocalInstallInitData
LocalInstallWithSD
OnlineInstallWithPkgs
ProxyInstall
LocalProxyInstall
LocalBrokerInstall
```

**STEP 5**: Let'r Rip

You can run `r <TestGroup>` to run any individual test group, or simply `r` to run the entire suite.

Start slowly by just running the Usage tests:

```
r Usage
```

Then trying something a bit more invovled, like installing a standalone broker:

```
r LocalBrokerInstall
```

Last, try the full regression test suite:

```
r
```

Somewhere toward the end of output (but possibly quite a few lines up from the very bottom), you'll hopefully see something like this:

```
ALL 196 tests PASSED.

That took about 0 hours 14 minutes 55 seconds.
```

For deeper investigation, view or edit the file pointed to by the symlink `/tmp/run_cli_tests.log`.

To get more detail on what tests are being executed for each test group, run `cdd` and then look at the `cli_tests.cfg` file.

## Qualifying a New Platform

Once you've bootstrapped a machine for a platform not yet in the `## Platforms` list above, formally qualify it before adding it there:

1. Get a clean full regression run using the `uat` alias (defined in `dev_env.sh` -- see the aliases list in the Tips section below):

   ```
   cdd
   uat
   ```

   Confirm the output includes a line like `ALL 287 tests PASSED.` (the exact test count may differ as the suite grows). If it doesn't, don't add the platform yet -- investigate the failure first.

2. Update the `## Platforms` list at the top of this document to add the new platform (and, if it's a new architecture or OS family, consider whether `bootstrap_test_machine.sh`'s `PlatformTagMap` -- see `./bootstrap_test_machine.sh -man` -- needs a new row so future machines on that platform auto-detect their `PlatformTag` instead of needing `-tag` on the command line).

## Appendix: What `bootstrap_test_machine.sh` Does Internally

This is what `bootstrap_test_machine.sh` does under the hood, for troubleshooting or if you want to run the steps by hand. The output should look something like this:

```
root@p4d-test-r9-01:/root/p4_binaries# curl -O https://ftp.perforce.com/perforce/r26.1/bin.linux26x86_64/p4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.1M  100 10.1M    0     0  8955k      0  0:00:01  0:00:01 --:--:-- 8955k
root@p4d-test-r9-01:/root/p4_binaries# curl -O https://ftp.perforce.com/perforce/r26.1/bin.linux26x86_64/p4d
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 17.5M  100 17.5M    0     0  13.4M      0  0:00:01  0:00:01 --:--:-- 13.4M
```

Then, the equivalent of the `p4 clone` and `swap_source.sh` commands:

```
root@p4d-test-r9-01:/opt/perforce/dev/bin# p4 -u ftp clone -p public.perforce.com:1666 -r test-install_sdp_dev
Server ftp-dvcs-1786465181 saved.
Cloning from 'public.perforce.com:1666'...
fetch load revisions 44 |finishing
fetch load integrations 25 |finishing
check revisions 100% |finishing
fetch archives 100% |finishing
commit revisions 100% |finishing
sync //test-install_sdp/dev/.p4ignore#1 - can't overwrite existing file /opt/perforce/dev/bin/.p4ignore
100% |finishing
13 change(s) containing a total of 44 file revision(s) were successfully fetched.

root@p4d-test-r9-01:/opt/perforce/dev/bin# ./swap_source.sh p4-sdp/dev_rebrand
No prior checkout found for p4-sdp/dev_rebrand; cloning fresh via remote spec p4-sdp_dev_rebrand (as ftp) ...
Server ftp-dvcs-1786465183 saved.
Cloning from 'public.perforce.com:1666'...
fetch load revisions 1227 |finishing
fetch load integrations 3935 |finishing
check revisions 100% |finishing
check integrations 100% |finishing
fetch archives 100% |finishing
commit revisions 100% |finishing
commit integrations 100% |finishing
sync //p4-sdp/dev_rebrand/.p4ignore#1 - can't overwrite existing file /opt/perforce/dev/sdp/.p4ignore
100% |finishing
68 change(s) containing a total of 1227 file revision(s) were successfully fetched.
sdp is now sourced from p4-sdp/dev_rebrand.
```

The "can't overwrite existing file .../.p4ignore" line in both cases is expected and benign -- it's why `bootstrap_test_machine.sh`/`swap_source.sh` fix up `.p4ignore` handling right after the clone.

## Tips

If you supplied your own Public Depot login, you'll eventually and occaisonally need to renew your Public Depot ticket.  Do do like this:

```
cds
p4 login -r origin    # <-- Supply the password for the P4USER you used with '-mode advanced' in STEP 3 above.
cdd
p4 login -r origin -s  # <-- This is just a test, you won't be prompted for the password again.
```

### Source Swapping

Once a machine is set up, `swap_source.sh` (in `/opt/perforce/dev/bin`) lets you change what either `/opt/perforce/dev/sdp` or `/opt/perforce/dev/bin` pulls from, without losing the ability to switch back:

```
cdd
./swap_source.sh p4-sdp/dev            # Switch the SDP to //p4-sdp/dev.
./swap_source.sh p4-sdp/main           # Switch the SDP to //p4-sdp/main.
./swap_source.sh p4-sdp/dev_rebrand    # Switch the SDP back to //p4-sdp/dev_rebrand.
./swap_source.sh test-install_sdp/main # Switch the test suite itself to //test-install_sdp/main.
```

The first time a given stream is requested on a machine, `swap_source.sh` does a fresh `p4 clone -r` for it (read-only, unless you set `P4_SWAP_USER` to your own Public Depot account first).  On later swaps back to a stream already used on that machine, it just moves the previously-parked local directory back into place -- no re-clone needed.  Parked directories are named `<sdp|bin>.<stream, with '/' as '_'>`, e.g. `sdp.p4-sdp_dev_rebrand` or `bin.test-install_sdp_main`, sitting alongside the active `sdp`/`bin` directory.

Be aware of handy aliases defined in `dev_env.sh` (loaded automatically when you login).  In this environment, you have the following aliases:

* `cds` is the alias to cd to the SDP setup dirextory, where the `install_sdp.sh` and `mkdirs.sh` scripts are versioned.
* `cdb` is the alias to cd to the SDP common/bin dirextory, where most SDP scripts are versioned.
* `cdd` is the alias to cd to the test operations directory, where the `r` utility (described above) lives.  Do a `cdd` before kicking off the test suite.
* `cdu` is the alias to cd to the SDP common/sdp_upgrade directory, the `sdp_upgrade.sh` script lives.
* `ec2id` is the alias to display the AWS EC2 instance id of the current machine (assuming your machine is deployed in AWS; it could be deployed anywhere).
* `uat` a quick alias that does several things:
  * Fetches latest files from this SDP install_sdp.sh regression test suite, by doing `cdd` then `p4 fetch`.
  * Fetches latest SDP files from the Public Depot by doing `cds` then `p4 fetch`.
  * Calls the `test_preflight.sh` script to make sure local files are aligned with version control and nothing is checked out.
  * If and only if the `test_preflight.sh` indicates all is well, kicks off the full regression test suite with the `r` script with no arguments.
* Scripts that have a `tss-` prefix are Test Support Scripts. These appear in the test operations directory, and are referenced in `cli_test.cfg`.


## SAFETY WARNING

**Use this procedure ONLY on an expendable virtual machine on which there is NO DATA OF ANY VALUE.**

The new SDP `install_sdp.sh` that this procedures tests is very safe.  However, the test suite for it is rather dangerous, especially the script clearly named `DANGER_CLEAN.sh`. This is called automatically, several times, when the full test suite is executed.