Welcome to the home of the Perforce Helix Installer project. The Helix Installer is an advanced method for installing Helix Core and the Server Deployment Package (SDP).
IT SHOULD ONLY BE USED IN A TOTALLY NEW HELIX CORE INSTALLATION!! IF IN DOUBT, DO NOT USE IT.
DO NOT USE THIS TOOL IF YOU HAVE EXISTING DATA IN YOUR HELIX REPOSITORY!! YOU RISK DELETING EVERYTHING!!
Do not use this tool to upgrade the SDP OR P4D!!
Perforce Helix is a unified platform for collaborative development, version management, and protection of intellectual property. It consists of an integrated ecosystem of proven, enterprise-grade systems for versioning, collaboration and code review, continuous integration and delivery, and other aspects of a DevOps pipeline.
This Helix Installer is capable of quickly transforming a vanilla Linux machine into a production-grade deployment of Helix Core. It also helps standardize deployment of a fleet of Helix Core servers across an enterprise-scale deployment of Helix Core.
For public cloud usage, the Helix Installer provides a way to build "Golden Copy" baseline server installations. For example, it plays a role in building Amazon Machine Images (AMIs). It is useful in other Linux installations, including public/private clouds, on-prem virtual or bare metal hardware Docker containers, etc. See Public Cloud Usage below.
This software SHOULD NEVER EXIST on a Production Perforce server. The is intended to quickly tear down and rebuild a fresh Perforce server (e.g. for operating test suites). It should not exist on a machine containing real Perforce data.
It is useful for doing an initial installation, e.g. to bootstrap a production machine with the SDP structure. After a successful install, the core reset_sdp.sh
script is moved aside to reset_sdp.sh.txt and the execute bit is removed. The script should be removed entirely before going live in production.
As of July, 2024:
This software is community supported. Evolution can also be driven by engaging Perforce Consulting. Please DO NOT contact Perforce Support for the Helix Installer, as it is not an officially supported product offering. However, the SDP that is installed with the Helix Installer is itself supported.
While not officially supported, the Helix Installer is actively maintained and absolutely integral to many processes that build upon the Helix Installer for higher levels of automation for various purposes such as full deployment automation of Helix Core servers. The code is exercised frequently and is relied on. Contact Perforce Helix Consulting for more information.
Note that the Helix Installer is bootstrapping software, for a single usage on any given server machine. Thus, there is reduced need for it to be officially supported.
The Helix Installer is installs the Server Deployment Package (SDP).
By default, the Helix Installer installs the latest mainline version of the SDP acquired from The Workshop over the public internet. It acquires the latest patch of the Helix Core binaries (currently Helix Core 2020.2) available from the Perforce FTP server.
This section is for advanced usage specific to developing the SDP and Helix Installer.
The Helix Installer can also clone the SDP project's dev
branch to get the very latest, but possibly not yet fully tested, software available in the SDP dev
branch. This is done using Helix native DVCS features (e.g. p4 clone
) to clone the SDP dev branch from The Workshop. Optionally a specified shelved changelist can unshelved over the dev branch (see SDP Dev Branch Testing below). As a general rule, it is best to stick with the default 'main' branch unless Perforce Consulting recommends using the dev branch for some reason (with any such reason being temporary and customer-specific).
The installation is suitable for:
-fast
option, which omits the optional package installation.The reset_sdp.sh
script is the only script that should be downloaded. It downloads everything it needs, including other scripts and files from this Helix Installer package, executables from the Perforce FTP site, and the SDP from The Workshop (the web front end to the Public Depot).
Once creating a fresh vanilla server machine for a supported platform, do this as the root user:
mkdir -p /hxdepots/reset
cd /hxdepots/reset
curl -L -O https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/main/src/reset_sdp.sh
chmod +x reset_sdp.sh
Read the documentation the manual page generated with -man
:
./reset_sdp.sh -man
And then, the big moment. Make it happen:
./reset_sdp.sh -fast 2>&1 | tee reset_sdp.log
A similar procedure works for Mac OSX, except that the standard Mac interface is used manually for creating the perforce
user account.
A configured installation can be done, allowing more options to be changed using a generated settings.cfg
file. Do like so:
su -
mkdir -p /hxdepots/reset
cd /hxdepots/reset
curl -L -O https://swarm.workshop.perforce.com/download/guest/perforce_software/helix-installer/main/src/reset_sdp.sh
chmod +x reset_sdp.sh
Generate a default config file:
./reset_sdp.sh -C > settings.cfg
Read the documentation the manual page generated with -man
:
./reset_sdp.sh -man
Edit the generated settings.cfg
file, changing the values as desired. And then run using that generated file:
./reset_sdp.sh -c settings.cfg
The Configured Installation method described above is the best way to bootstrap a production installation. If used for this purpose, and if separate storage mounts are to be used for optimal and/or at-scale installations, these mounts should be configured (e.g. AWS CloudFormation Templates, or Terraform) first:
/hxdepots
- Used for versioned files and checkpoints, etc./hxmetadata
- Used for live and offline databases, etc. /hxlogs
- Used for script and application logs, active journal, etc.These should all be configured as xfs
filesystems (or alternately ext4
).
In addition, acquire a license file from sales@perforce.com for evaluation or license purchase.
See the Volume Layout and Hardware section of the SDP Guide for Unix for more information.
The Helix Installer can be used to test SDP changes shelved to the SDP dev branch in The Workshop. The following example illustrates testing a change in shelved changelist 23123:
su -
mkdir -p /hxdepots/reset
cd /hxdepots/reset
curl -L -O https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/main/src/reset_sdp.sh
chmod +x reset_sdp.sh
./reset_sdp.sh -b dev,@23123 2>&1 | tee reset_sdp.CL23123.log
After the first test, an iterative test cycle may follow on the same shelved changelist. For each test iteration, the shelved changelist is first updated in the workspace from which the change was originally shelved, e.g. with a command like p4 shelve -f -c 23123
.
Then a new test can be done by calling reset_sdp.sh
with the same arguments. The script will re-install the SDP cleanly, and then re-apply the updated shelved changelist.
See the Helix Installer Release Notes.
Helix Installer 2024.1 is available today. It installs the following Helix Components, all fully configured an integrated:
The Helix Installer 2024.1 version handles all aspects of installation. It does the following:
perforce
OS user, if needed.Following installation, it also does the following to be more convenient for demos, and also give a more production-like feel:
perforce
user sudo access (optionally limited sudo access).~perforce/.bash_profile
and .bashrc
files (if they do not already exist).p4d
and other binaries directly from the Perforce FTP server).firewalld
type firewalls only).The Helix Installer Test Suite uses Docker virtualization technology. The test suite can be used for standalone demonstrations of the Helix Installer or the SDP.
Helix Installer 2024.1 Platform Support notes:
Distro | Support | Notes |
---|---|---|
Amazon Linux 2 | L | Limited support. Known to work. This will go EOL on June 30, 2025, at which point it will be Unsupported. |
RHEL/CentOS 7.9+ | A | Unsupported. This went EOL on June 30, 2024. |
RHEL/Rocky Linux 8.9+ | A | Supported. Also supported: Rocky Linux 8 Security Hardened by Perforce OpenLogic. |
RHEL/Rocky Linux 9.0+ | M | Supported. Also supported: Rocky Linux 9 Standard by Perforce OpenLogic. |
Ubuntu 20.04 LTS Release (Focal Fossa) | A | Supported |
Ubuntu 22.04 LTS Release (Jammy Jellyfish) | A | Supported |
Fedora | U | Unsupported and discouraged for production usage, as it is a testbed OS. |
CentOS 8 | U | Unsupported and discouraged for production usage, as it is a testbed OS. |
Ubuntu 18.04 LTS Release (Bionic Beaver) | U | Unsupported. EOL. The default shell in Bionic causes the Helix Installer to fail. |
Ubuntu 16.04 LTS Release (Xenial) | U | Unsupported. EOL. |
OSX 10.14 Mojave | L | Limited support for Mac OSX/Darwin. Does not create 'perforce' OS user, other limitations. Note: There is no plan to support newer versions OSX due to lack of need and OS changes. |
SuSE 15 | L | Known to work at least once. No ongoing testing; could be added if there is sufficient need. |
SuSE 12 | U | Unsupported and EOL. Known to work at least once, no ongoing testing. |
CentOS/RHEL 6.9 | U | Unsupported and EOL, but known to work. |
As of July 2024 this project is functional and active, and has been actively used since its inception in April 2015. It has been actively maintained to keep pace with SDP changes up to and including the SDP 2024.1 release.
For building out cloud environments, some customers build "Golden Copy" images based on the Helix Installer.
AWS CloudFormation templates as well as cloud-agnostic Terraform and similar technologies can be used to build baseline infrastructure in any cloud, private or public. Then the Helix Installer transforms a baseline machine (configured with the 3 /hx*
storage volumes) into a baseline Helix machine to be a master, edge server, etc. Then after data is loaded, a new "local baseline" machine image with local Helix data, potentially Terabytes, can be added.
To further simplify this, publicly available Amazon Machine Images (AMIs) based on this Helix Installer are available in AWS and Azure marketplaces. For more information, see the Perforce Enhanced Studio Pack.
While using the latest publicly available versions of software is generally reliable and common practice, there are usage scenario where guaranteed reliability is required, and/or public public internet access is not available. For these scenarios, the Helix installer sports a -local
option for Local Mode installation.
For a guaranteed reliable install that does not rely on public internet access, the Helix Installer also supports installation with fixed (and possibly older) versions of the software it needs, specifically:
To prepare for stable installation, a local tarball must first be created in an environment that has public internet access. This is done like so:
As root
, download the SDP tarball.
mkdir -p /hxdepots/downloads
cd /hxdepots/downloads
curl -L -o sdp.Unix.tgz https://swarm.workshop.perforce.com/downloads/guest/perforce_software/sdp/downloads/sdp.Unix.tgz?v=%2351
cd ..
tar -xzf downloads/sdp.Unix.tgz
The odd-looking text at the end of the URL ?v=%2351
is a version specifier for version 51 (not a typo).
Execute commands like these samples to get the binaries. This gets Helix Core 2024.1 binaries. Tweak the URL as desired. For example, change r24.1 to r23.2 in the examples below to get the 2023.2 binaries.
cd /hxdepots/sdp/helix_binaries
curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4
curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4d
curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4broker
curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4p
chmod +x p4*
Only the latest patch for a specified major version (e.g. r24.1) can be obtained from the Perforce FTP server. There is no standard way to get an earlier patch, as new patches supersede older ones.
Then, get the tarball for the Helix Installer (2024.1) onto the machine in which you intend to perform the stable installation.
cd /hxdepots/downloads
curl -L -o helix_installer.tgz https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/downloads/helix_installer.tgz?v=%2322
tar -xzf helix_installer.tgz
rm -rf /hxdepots/reset
cp -pr hi/src ../reset
The odd-looking text at the end of the URL ?v=%2322
is a version specifier for version 22 (not a typo).
Acquire the Sample Depot training data set from the Perforce FTP server. This can be skipped if you plan to use the -no_sd
(No Sample Depot) option to the Helix Installer.
cd /hxdepots/downloads
curl -O http://ftp.perforce.com/perforce/tools/sampledepot.tar.gz
With the steps above completed to prepare local data, the entire /hxdepots tree, including the Helix Installer, SDP, Helix binaries, and optional Sample Depot.
cd /
tar -czf hxdepots.tgz hxdepots
The hxdepots.tgz
tarball can then be transferred (over air gap network if necessary) to the machine on which stable installation is to be performed. On that machine, extract such that the structure contains:
/hxdepots/reset
/hxdepots/downloads
The precise commands to transfer files and untar on the other machine will vary depending on whether mount points are involved, what directories may already exist on the target machine, and your file transfer mechanism. One of the commands need will be tar -xzf hxdepots.tgz
to create the folder structure that must end up in /hxdepots
. It may be necessary to copy that to some other temporary location first and use rsync or similar to populate /hxdepots
.
When done, you can execute the Helix Installer per usual instructions above, except that you will use the -local
option, and and avoid using -B
(blast) option, as that will require rebuilding the /hxdepots
structure on the air gapped machine.
In late 2021, a Perforce Consulting customer embarked on a project to automate deployment of a new Helix Core edge or replica server to such a degree that someone with limited knowledge of Perforce Helix could deploy a new server in an enterprise, with only minimal inputs.
A pre-requisite was a baseline VM image built with just the basics:
perforce
OS user with limited sudo.For ease of use, the user interface chosen was simply the existing and familiar mkrep.sh
script in the SDP, which already captures the information needed:
The standard mkrep.sh
enables custom post-processing programs to be called with hooks. The hooks are used so mkrep.sh
post-processing picks up where mkrep.sh
leaves off, doing everything that mkrep.sh
doesn't, including:
rotate_journal.sh
to get replica metadata configuration into a numbered journal file.load_checkpoint.sh
to load the latest checkpoint and subsequent journals on the new machine.p4verify.sh
script to schedule all archives for transfer.The implementation uses the "Tight Ship" features of the Helix Management System (HMS) to store key data bits enabling operation behind a restrictive corporate firewall, such as
settings.cfg
file for the Helix Installer with local preferences for mount points, etc.Some enterprise customers take full advantage of the Helix Installer for public cloud and hybrid environments that blend public cloud, private cloud, and on-prem systems. The Helix Installer integrates into a larger technology stack to build out sophisticated global topologies. The key build blocks are the Helix Installer (reset_sdp.sh
) and the SDP mkrep.sh
, which together can easily build out a sophisticated global Helix Core topology in minutes -- longer of course when you add in Terabytes of data. At the top of that food chain, the Helix Management System (HMS) adds a layer of global topology awareness.
A slightly modified Helix Installer is an integral component of the Lab Engine behind the Battle School Workshop training course. The Battle School Workshop Lab Engine quickly tears down and spins up a fresh, enterprise-style Helix installation -- and then breaks it in order to simulate real-world problems that students learn to identify and fix. The Battle School version of the Helix Installer (not available for public consumption) adds a simulated global topology with replicas and edge servers.
Note: This is an older example, as there is no longer a need for custom automation to support CBD workflows in Helix Core; this example was developed circa 2013.
The Component Based Development (CBD) system, another project in The Workshop, uses this Helix Installer as part of its automated test suite. In conjunction with Vagrant and Virtual Box, which build a vanilla VM from the ground up, the CBD Test Suite uses the Helix Installer to deploy a Perforce server, complete with Sample Depot test data set, plus Python and P4Python needed to test the CBD scripts.
Please [contact Perforce Helix Consulting Services] (mailto:consulting-helix-core@perforce.com) for more information.
# Perforce Helix Installer ## Welcome Welcome to the home of the Perforce Helix Installer project. The Helix Installer is an advanced method for installing Helix Core and the [Server Deployment Package (SDP)](https://swarm.workshop.perforce.com/projects/perforce-software-sdp). IT SHOULD ONLY BE USED IN A TOTALLY NEW HELIX CORE INSTALLATION!! IF IN DOUBT, DO NOT USE IT. DO NOT USE THIS TOOL IF YOU HAVE EXISTING DATA IN YOUR HELIX REPOSITORY!! YOU RISK DELETING EVERYTHING!! **Do not use this tool to upgrade the SDP OR P4D!!** Perforce [Helix](http://www.perforce.com/helix) is a unified platform for collaborative development, version management, and protection of intellectual property. It consists of an integrated ecosystem of proven, enterprise-grade systems for versioning, collaboration and code review, continuous integration and delivery, and other aspects of a DevOps pipeline. This **Helix Installer** is capable of quickly transforming a vanilla Linux machine into a production-grade deployment of Helix Core. It also helps standardize deployment of a fleet of Helix Core servers across an enterprise-scale deployment of Helix Core. For public cloud usage, the Helix Installer provides a way to build "Golden Copy" baseline server installations. For example, it plays a role in building Amazon Machine Images (AMIs). It is useful in other Linux installations, including public/private clouds, on-prem virtual or bare metal hardware Docker containers, etc. See Public Cloud Usage below. ## Safety WARNING This software ***SHOULD NEVER EXIST*** on a Production Perforce server. The is intended to quickly tear down and rebuild a fresh Perforce server (e.g. for operating test suites). ***It should not exist on a machine containing real Perforce data.*** It is useful for doing an initial installation, e.g. to bootstrap a production machine with the SDP structure. After a successful install, the core `reset_sdp.sh` script is moved aside to reset_sdp.sh.txt and the execute bit is removed. The script should be removed entirely before going live in production. ## Support Status As of July, 2024: This software is **community supported**. Evolution can also be driven by engaging Perforce Consulting. Please DO NOT contact Perforce Support for the Helix Installer, as it is not an officially supported product offering. However, the SDP that is installed with the Helix Installer is itself supported. While not officially supported, the Helix Installer is actively maintained and absolutely integral to many processes that build upon the Helix Installer for higher levels of automation for various purposes such as full deployment automation of Helix Core servers. The code is exercised frequently and is relied on. Contact [Perforce Helix Consulting](mailto:consulting-helix-core@perforrce.com) for more information. Note that the Helix Installer is bootstrapping software, for a single usage on any given server machine. Thus, there is reduced need for it to be officially supported. ## Relationship to Server Deployment Package (SDP) The Helix Installer is installs the [Server Deployment Package (SDP)](https://swarm.workshop.perforce.com/projects/perforce-software-sdp). By default, the Helix Installer installs the latest mainline version of the SDP acquired from The Workshop over the public internet. It acquires the latest patch of the Helix Core binaries (currently Helix Core 2020.2) available from the Perforce FTP server. ## SDP Dev Branch Install This section is for advanced usage specific to developing the SDP and Helix Installer. The Helix Installer can also clone the SDP project's `dev` branch to get the very latest, but possibly not yet fully tested, software available in the SDP `dev` branch. This is done using Helix native DVCS features (e.g. `p4 clone`) to clone the SDP dev branch from The Workshop. Optionally a specified shelved changelist can unshelved over the dev branch (see SDP Dev Branch Testing below). As a general rule, it is best to stick with the default 'main' branch unless Perforce Consulting recommends using the dev branch for some reason (with any such reason being temporary and customer-specific). ## Helix Installer Uses The installation is suitable for: * Test suites for Helix server trigger and broker filter scripts deployed in enterprise environments. For example, the [CBD](https://swarm.workshop.perforce.com/projects/perforce-software-cbd) test suite uses the Helix Installer to quickly spin up a test environment for server and triggers * Demonstrations of Helix Core * Proof of Concept (PoC) Deployments * Training * Evaluation and Pilot projects. Because the Helix Installer uses the SDP, you can **transition with minimal friction from a demo-grade installation to an enterprise-grade installation**. * **Bootstrapping production installation quickly**. * Testing SDP changes shelved to the dev branch in The Workshop (see SDP Dev Branch Testing below). Features --- * **Easy**. Provides reasonable defaults for port numbers, passwords, data storage areas, and all inputs that would otherwise require human interaction. * **Comprehensive**. Installs P4D and P4Broker, SDP, including Python and P4Python, with appropriate shell environment settings to use them all immediately. And more to come -- future versions may include Helix Swarm, etc. * **Fast** - Runs in about 2 minutes with the `-fast` option, which omits the optional package installation. * **Simplifies Enterprise Deployment**. While optimized for getting going quickly, the Helix Installer is designed to enable a fast path from a Proof of Concept (PoC) to small-scale production deployment following the Helix evaluation, with only a nominal hop from there to a large-scale global enterprise deployment. Global Enterprise deployments require additional planning and custom configuration based on the specific global topology. * Optionally pre-loads the [Perforce Sample Depot](https://portal.perforce.com/s/article/2439), making it instantly ready for training and evaluation purposes. ### Instructions and Usage Example #### Fast Demo Installations The `reset_sdp.sh` script is the only script that should be downloaded. It downloads everything it needs, including other scripts and files from this Helix Installer package, executables from the Perforce FTP site, and the SDP from The Workshop (the web front end to the Public Depot). Once creating a fresh vanilla server machine for a supported platform, do this as the *root* user: ``` mkdir -p /hxdepots/reset cd /hxdepots/reset curl -L -O https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/main/src/reset_sdp.sh chmod +x reset_sdp.sh ``` Read [the documentation the manual page](https://swarm.workshop.perforce.com/view/guest/perforce_software/helix-installer/main/doc/reset_sdp.command_summary.txt) generated with `-man`: ``` ./reset_sdp.sh -man ``` And then, the big moment. Make it happen: ``` ./reset_sdp.sh -fast 2>&1 | tee reset_sdp.log ``` A similar procedure works for Mac OSX, except that the standard Mac interface is used manually for creating the `perforce` user account. #### Configured Installation A configured installation can be done, allowing more options to be changed using [a generated `settings.cfg`](https://swarm.workshop.perforce.com/view/guest/perforce_software/helix-installer/main/doc/settings.cfg) file. Do like so: ``` su - mkdir -p /hxdepots/reset cd /hxdepots/reset curl -L -O https://swarm.workshop.perforce.com/download/guest/perforce_software/helix-installer/main/src/reset_sdp.sh chmod +x reset_sdp.sh ``` Generate a default config file: ``` ./reset_sdp.sh -C > settings.cfg ``` Read [the documentation the manual page](https://swarm.workshop.perforce.com/view/guest/perforce_software/helix-installer/main/doc/reset_sdp.command_summary.txt) generated with `-man`: ``` ./reset_sdp.sh -man ``` Edit the [generated `settings.cfg` file](https://swarm.workshop.perforce.com/projects/perforce_software-helix-installer/view/main/doc/settings.cfg), changing the values as desired. And then run using that generated file: ``` ./reset_sdp.sh -c settings.cfg ``` #### Bootstrapping Production The Configured Installation method described above is the best way to bootstrap a production installation. If used for this purpose, and if separate storage mounts are to be used for optimal and/or at-scale installations, these mounts should be configured (e.g. AWS CloudFormation Templates, or Terraform) first: * `/hxdepots` - Used for versioned files and checkpoints, etc. * `/hxmetadata` - Used for live and offline databases, etc. * `/hxlogs` - Used for script and application logs, active journal, etc. These should all be configured as `xfs` filesystems (or alternately `ext4`). In addition, acquire a license file from sales@perforce.com for evaluation or license purchase. See the **Volume Layout and Hardware** section of [the SDP Guide for Unix](https://swarm.workshop.perforce.com/view/guest/perforce_software/sdp/main/doc/SDP_Guide.Unix.html#_volume_layout_and_hardware) for more information. #### SDP Dev Branch Testing The Helix Installer can be used to test SDP changes shelved to the SDP dev branch in The Workshop. The following example illustrates testing a change in shelved changelist 23123: ``` su - mkdir -p /hxdepots/reset cd /hxdepots/reset curl -L -O https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/main/src/reset_sdp.sh chmod +x reset_sdp.sh ./reset_sdp.sh -b dev,@23123 2>&1 | tee reset_sdp.CL23123.log ``` After the first test, an iterative test cycle may follow on the same shelved changelist. For each test iteration, the shelved changelist is first updated in the workspace from which the change was originally shelved, e.g. with a command like ```p4 shelve -f -c 23123```. Then a new test can be done by calling `reset_sdp.sh` with the same arguments. The script will re-install the SDP cleanly, and then re-apply the updated shelved changelist. ## Helix Installer Release Notes See the [Helix Installer Release Notes](https://swarm.workshop.perforce.com/files/guest/perforce_software/helix-installer/main/doc/ReleaseNotes.md). ## Helix Installer Roadmap ### Helix Installer 2024.1 Helix Installer 2024.1 is available today. It installs the following Helix Components, all fully configured an integrated: * **P4D** 2024.1 - the Helix Versioning Engine, the core of Helix (with or without SSL enabled). * **P4Broker** 2024.1 - the Helix Broker, enhances custom automation capabilities, simplifies maintenances. * **SDP** - Server Deployment Package (SDP) 2024.1, latest release from the SDP mainline. * **Sample Depot** data set for training, evaluation, and demo, and pilot projects. The Helix Installer 2024.1 version handles all aspects of installation. It does the following: * Creates the `perforce` OS user, if needed. * Creates the home directory for the perforce user, if needed. * Adds recommended OS packages for yum, apt, and zypper package management systems. * Adds package for Python/P4Python for yum and apt package management systems. Following installation, it also does the following to be more convenient for demos, and also give a more production-like feel: * Grants the `perforce` user sudo access (optionally limited sudo access). * Creates default `~perforce/.bash_profile` and `.bashrc` files (if they do not already exist). * Connects to the [Perforce Package Repository](https://www.perforce.com/perforce-packages) (APT and YUM only) to enable [Helix Swarm](https://www.perforce.com/products/helix-swarm) installation. (Note this script acquires `p4d` and other binaries directly from the Perforce FTP server). * Makes firewall adjustments (`firewalld` type firewalls only). * Makes init system adjustments, for both SysV and systemd init mechanisms. * Makes SELinux/systemd for systemd systems. ## Test Suite The Helix Installer Test Suite uses Docker virtualization technology. The test suite can be used for standalone demonstrations of the Helix Installer or the SDP. ## Platform Support Helix Installer 2024.1 Platform Support notes: Distro | Support | Notes --- | --- | --- Amazon Linux 2| L | Limited support. Known to work. This will go EOL on June 30, 2025, at which point it will be Unsupported. RHEL/CentOS 7.9+ | A | **Unsupported**. This went EOL on June 30, 2024. RHEL/Rocky Linux 8.9+ | A | **Supported**. Also supported: [Rocky Linux 8 Security Hardened](https://aws.amazon.com/marketplace/pp/prodview-fs55cawwcx75c) by Perforce OpenLogic. RHEL/Rocky Linux 9.0+ | M | **Supported**. Also supported: [Rocky Linux 9 Standard](https://aws.amazon.com/marketplace/pp/prodview-gjbzdjrmtuhta) by Perforce OpenLogic. Ubuntu 20.04 LTS Release (Focal Fossa)| A| **Supported** Ubuntu 22.04 LTS Release (Jammy Jellyfish)| A| **Supported** Fedora | U | Unsupported and discouraged for production usage, as it is a testbed OS. CentOS 8 | U | Unsupported and discouraged for production usage, as it is a testbed OS. Ubuntu 18.04 LTS Release (Bionic Beaver)| U| Unsupported. EOL. The default shell in Bionic causes the Helix Installer to fail. Ubuntu 16.04 LTS Release (Xenial)| U | Unsupported. EOL. OSX 10.14 Mojave | L | Limited support for Mac OSX/Darwin. Does not create 'perforce' OS user, other limitations. Note: There is no plan to support newer versions OSX due to lack of need and OS changes. SuSE 15 | L | Known to work at least once. No ongoing testing; could be added if there is sufficient need. SuSE 12 | U | Unsupported and EOL. Known to work at least once, no ongoing testing. CentOS/RHEL 6.9 | U | Unsupported and EOL, but known to work. ### Support Codes: * **A**: Actively tested and included in the automated regression test suite; works reliably. * **M**: Active manual testing done in this environment. * **I**: Infrequent/occasional manual testing done in this environment. * **L**: Limited support * **U**: Unsupported ## Project Status As of July 2024 this project is functional and active, and has been actively used since its inception in April 2015. It has been actively maintained to keep pace with SDP changes up to and including the SDP 2024.1 release. ## Public Cloud Usage For building out cloud environments, some customers build "Golden Copy" images based on the Helix Installer. AWS CloudFormation templates as well as cloud-agnostic Terraform and similar technologies can be used to build baseline infrastructure in any cloud, private or public. Then the Helix Installer transforms a baseline machine (configured with the 3 `/hx*` storage volumes) into a baseline Helix machine to be a master, edge server, etc. Then after data is loaded, a new "local baseline" machine image with local Helix data, potentially Terabytes, can be added. To further simplify this, publicly available Amazon Machine Images (AMIs) based on this Helix Installer are available in AWS and Azure marketplaces. For more information, see the [Perforce Enhanced Studio Pack](https://www.perforce.com/perforce-and-cloud). ## Stable Installation While using the latest publicly available versions of software is generally reliable and common practice, there are usage scenario where guaranteed reliability is required, and/or public public internet access is not available. For these scenarios, the Helix installer sports a `-local` option for Local Mode installation. For a guaranteed reliable install that does not rely on public internet access, the Helix Installer also supports installation with fixed (and possibly older) versions of the software it needs, specifically: * Server Deployment Package (SDP), 2024.1. * Get Helix Core binaries, 2024.1. * Helix Installer, 2024.1. To prepare for stable installation, a local tarball must first be created in an environment that has public internet access. This is done like so: ### Get a fixed version of the SDP. As `root`, download the SDP tarball. ``` mkdir -p /hxdepots/downloads cd /hxdepots/downloads curl -L -o sdp.Unix.tgz https://swarm.workshop.perforce.com/downloads/guest/perforce_software/sdp/downloads/sdp.Unix.tgz?v=%2351 cd .. tar -xzf downloads/sdp.Unix.tgz ``` The odd-looking text at the end of the URL `?v=%2351` is a version specifier for version 51 (not a typo). ### Get the Helix Binaries Execute commands like these samples to get the binaries. This gets Helix Core 2024.1 binaries. Tweak the URL as desired. For example, change r24.1 to r23.2 in the examples below to get the 2023.2 binaries. ``` cd /hxdepots/sdp/helix_binaries curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4 curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4d curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4broker curl -O http://ftp.perforce.com/perforce/r24.1/bin.linux26x86_64/p4p chmod +x p4* ``` Only the latest patch for a specified major version (e.g. r24.1) can be obtained from the Perforce FTP server. There is no standard way to get an earlier patch, as new patches supersede older ones. ### Get Helix Installer Then, get the tarball for the Helix Installer (2024.1) onto the machine in which you intend to perform the stable installation. ``` cd /hxdepots/downloads curl -L -o helix_installer.tgz https://swarm.workshop.perforce.com/downloads/guest/perforce_software/helix-installer/downloads/helix_installer.tgz?v=%2322 tar -xzf helix_installer.tgz rm -rf /hxdepots/reset cp -pr hi/src ../reset ``` The odd-looking text at the end of the URL `?v=%2322` is a version specifier for version 22 (not a typo). ### Get the Sample Depot Acquire the Sample Depot training data set from the Perforce FTP server. This can be skipped if you plan to use the `-no_sd` (No Sample Depot) option to the Helix Installer. ``` cd /hxdepots/downloads curl -O http://ftp.perforce.com/perforce/tools/sampledepot.tar.gz ``` ### Make a Combined Tarball With the steps above completed to prepare local data, the entire /hxdepots tree, including the Helix Installer, SDP, Helix binaries, and optional Sample Depot. ``` cd / tar -czf hxdepots.tgz hxdepots ``` The `hxdepots.tgz` tarball can then be transferred (over air gap network if necessary) to the machine on which stable installation is to be performed. On that machine, extract such that the structure contains: * `/hxdepots/reset` * `/hxdepots/downloads` The precise commands to transfer files and untar on the other machine will vary depending on whether mount points are involved, what directories may already exist on the target machine, and your file transfer mechanism. One of the commands need will be `tar -xzf hxdepots.tgz` to create the folder structure that must end up in `/hxdepots`. It may be necessary to copy that to some other temporary location first and use rsync or similar to populate `/hxdepots`. When done, you can execute the Helix Installer per usual instructions above, except that you will use the `-local` option, and and avoid using `-B` (blast) option, as that will require rebuilding the `/hxdepots` structure on the air gapped machine. ## Case Studies ### Case Study: Magic Deployment Automation In late 2021, a Perforce Consulting customer embarked on a project to automate deployment of a new Helix Core edge or replica server to such a degree that someone with limited knowledge of Perforce Helix could deploy a new server in an enterprise, with only minimal inputs. A pre-requisite was a baseline VM image built with just the basics: * Storage configured with needed volumes. * A `perforce` OS user with limited sudo. * Pre-configured SSH keys for the Perforce user. For ease of use, the user interface chosen was simply the existing and familiar `mkrep.sh` script in the SDP, which already captures the information needed: * The new replica type (edge, HA, HA of edge, etc. and later proxies). * A geographic site tag. * the new machine name. The standard `mkrep.sh` enables custom post-processing programs to be called with hooks. The hooks are used so `mkrep.sh` post-processing picks up where `mkrep.sh` leaves off, doing everything that `mkrep.sh` doesn't, including: * Calling `rotate_journal.sh` to get replica metadata configuration into a numbered journal file. * Calling the Helix Installer to transform the baseline VM into a Helix Core server fit for the intended purpose. * Copying the latest checkpoint and subsequent journals from the master to the new machine. * Calling `load_checkpoint.sh` to load the latest checkpoint and subsequent journals on the new machine. * Calling the new fast `p4verify.sh` script to schedule all archives for transfer. The implementation uses the "Tight Ship" features of the [Helix Management System (HMS)](https://swarm.workshop.perforce.com/projects/perforce_software-hms) to store key data bits enabling operation behind a restrictive corporate firewall, such as * A local `settings.cfg` file for the Helix Installer with local preferences for mount points, etc. * The local version of the SDP. * Helix Installer. * Helix Core binaries. ### Case Study: Enterprise Environment Usage Some enterprise customers take full advantage of the Helix Installer for public cloud and hybrid environments that blend public cloud, private cloud, and on-prem systems. The Helix Installer integrates into a larger technology stack to build out sophisticated global topologies. The key build blocks are the Helix Installer (`reset_sdp.sh`) and the SDP `mkrep.sh`, which together can easily build out a sophisticated global Helix Core topology in minutes -- longer of course when you add in Terabytes of data. At the top of that food chain, the [Helix Management System (HMS)](https://swarm.workshop.perforce.com/projects/perforce_software-hms) adds a layer of global topology awareness. ### Case Study: Sample Usage of Helix Installer in Battle School Workshop A slightly modified Helix Installer is an integral component of the Lab Engine behind the Battle School Workshop training course. The Battle School Workshop Lab Engine quickly tears down and spins up a fresh, enterprise-style Helix installation -- and then breaks it in order to simulate real-world problems that students learn to identify and fix. The Battle School version of the Helix Installer (not available for public consumption) adds a simulated global topology with replicas and edge servers. ### Case Study: Sample Usage of Helix Installer in CBD Test Suite *Note: This is an older example, as there is no longer a need for custom automation to support CBD workflows in Helix Core; this example was developed circa 2013.* The [Component Based Development (CBD)](https://swarm.workshop.perforce.com/projects/perforce-software-cbd) system, another project in The Workshop, uses this Helix Installer as part of its automated test suite. In conjunction with Vagrant and Virtual Box, which build a vanilla VM from the ground up, the [CBD Test Suite](https://swarm.workshop.perforce.com/files/guest/perforce_software/cbd/main/test/auto_test_cbd_vagrant.sh) uses the Helix Installer to deploy a Perforce server, complete with Sample Depot test data set, plus Python and P4Python needed to test the CBD scripts. ## Contact Us Please [contact Perforce Helix Consulting Services] (mailto:consulting-helix-core@perforce.com) for more information.
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#39 | 30497 | C. Thomas Tyler |
Released SDP 2024.1.30495 (2024/07/23). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#38 | 30414 | C. Thomas Tyler |
Released SDP 2024.1.30412 (2024/06/13). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#37 | 30301 | C. Thomas Tyler |
Released SDP 2023.2.30299 (2024/05/09). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#36 | 30046 | C. Thomas Tyler |
Released SDP 2023.1.29958 (2023/12/01). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#35 | 29960 | C. Thomas Tyler |
Released SDP 2023.1.29958 (2023/12/01). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#34 | 29913 | Robert Cowham |
Add warnings at the top that this is not preferred method - too dangerous for inexperiecned users. To be revisited later. |
||
#33 | 29707 | C. Thomas Tyler |
Released SDP 2023.1.29698 (2023/07/11). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#32 | 29618 | C. Thomas Tyler |
Released SDP 2023.1.29616 (2023/05/25). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#31 | 29450 | C. Thomas Tyler |
Released SDP 2022.2.29448 (2023/02/27). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#30 | 29405 | C. Thomas Tyler |
Released SDP 2022.2.29403 (2023/02/06). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#29 | 28931 | C. Thomas Tyler |
Released SDP 2022.1.28929 (2022/07/05). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#28 | 28863 | C. Thomas Tyler |
Released SDP 2022.1.28834 (2022/05/27). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#27 | 28665 | C. Thomas Tyler |
Released SDP 2022.1.28663 (2022/03/08). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#26 | 28415 | C. Thomas Tyler |
Released SDP 2021.4.28409 (2021/11/24). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#25 | 28129 | C. Thomas Tyler |
Released SDP 2021.3.28126 (2021/10/24). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#24 | 27427 | C. Thomas Tyler |
Released SDP 2021.2.27425 (2021/02/09). Copy Up using 'p4 copy -r -b perforce_software-helix-installer-dev'. |
||
#23 | 26976 | C. Thomas Tyler | Minor doc tweaks to README.md | ||
#22 | 26975 | C. Thomas Tyler | Released HelixInstaller 2020.2.26973 (2020/12/01). | ||
#21 | 26031 | C. Thomas Tyler | Released Helix Installer 2019.4.26026 (2019/08/22). | ||
#20 | 25999 | C. Thomas Tyler | Released HelixInstaller 2019.4.25995 (2019/08/20). | ||
#19 | 25871 | C. Thomas Tyler | Released Helix Installer 2019.3.25869 (2019/07/23) | ||
#18 | 25822 | C. Thomas Tyler | Released HelixInstaller/MultiArch/2019.3/25820 (2019/07/11). | ||
#17 | 25725 | C. Thomas Tyler | Released HelixInstaller/MultiArch/2019.2/25723 (2019/06/19). | ||
#16 | 25600 | C. Thomas Tyler | Released HelixInstaller 2019.2.25598. | ||
#15 | 25315 | C. Thomas Tyler |
Released Rev. HelixInstaller/MultiArch/2019.1/25313 (2019/03/07). |
||
#14 | 25284 | C. Thomas Tyler |
Released Rev. HelixInstaller/MultiArch/2019.1/25282 (2019/03/06). |
||
#13 | 25260 | C. Thomas Tyler |
Released Rev. HelixInstaller/MultiArch/2019.1/25250 (2019/03/02). |
||
#12 | 24865 | C. Thomas Tyler | Released HelixInstaller v2018.2.24862. | ||
#11 | 23875 | C. Thomas Tyler | Released HelixInstaller/MultiArch/2018.1/23873. | ||
#10 | 23872 | C. Thomas Tyler | Release HelixInstaller/MultiArch/2018.1/23869. | ||
#9 | 23868 | C. Thomas Tyler | Released HelixInstaller/MultiArch/2018.1/23866 (2018/04/11). | ||
#8 | 23608 | C. Thomas Tyler |
Added 'yum install -y gcc gcc-c++' to avoid getting an error that tells you to do that anyway. |
||
#7 | 23283 | C. Thomas Tyler | Released Helix Installer 2017.4.23281 | ||
#6 | 23009 | C. Thomas Tyler | Tweaked LICENSE copyright year and README.md. | ||
#5 | 21671 | C. Thomas Tyler |
Varios updates to the README.md file: * Corrected to use 'curl' rather than 'p4 print' to get the reset_sdp.sh script, since the 'p4' won't exist on a vanilla CentOS/RHEL box. * Updated to reflect modern SDP, using /hxdepots rather than /depotdata. * Updated Project Status to note that support for Swarm and Git Fusion have been deferred. |
||
#4 | 19266 | C. Thomas Tyler | Minor corrections and typo fixes. | ||
#3 | 19265 | C. Thomas Tyler |
Updated README.md for first formal release. Added Version and doc/ReleaseNotes.md files. |
||
#2 | 19261 | C. Thomas Tyler | Accounted for move to //guest/perforce_software from //guest/tom_tyler. | ||
#1 | 19230 | C. Thomas Tyler |
Populate -o //guest/tom_tyler/helix-installer/... //guest/perforce_software/helix-installer/.... |
||
//guest/tom_tyler/helix-installer/main/README.md | |||||
#5 | 16848 | C. Thomas Tyler | Fixed a typo. | ||
#4 | 16672 | C. Thomas Tyler | Added clear step-by-step usage instructions. | ||
#3 | 16669 | C. Thomas Tyler | Updated README.md file for Helix Installer Project. | ||
#2 | 16410 | C. Thomas Tyler | Updated supported versions. | ||
#1 | 15829 | C. Thomas Tyler |
Moved Helix Installer to new Workshop Standard for personal projects. |
||
//guest/tom_tyler/sw/main/hi/README.md | |||||
#1 | 12335 | C. Thomas Tyler | Copied README.me up to main. | ||
//guest/tom_tyler/sw/dev/hi/README.md | |||||
#1 | 12333 | C. Thomas Tyler | Bootstrapped doc file. |