This document describes the process used to release the Server Deployment Package (SDP). This is intended for staff developing the SDP.
A dedicated SDP workspace should be used for releasing the SDP, separate from any you might use for SDP development. This dedicated release workspace should not contain any cruft files such as editor temp files. The release process mechanisms prevents known-cruft files from being added such as *~
files and .DS_Store
files on Mac. Using a clean dedicated release workspace guarantees no local excess files can sneak into the packaging process.
Use this JIRA Project to plan your work. This JIRA instance is only accessible with Perforce corporate SSO.
The SDP product owner determines what the release will be. That is 2023.1.3 (aka 2023.1 Patch 3).
The SDP shares the same major version as Helix Core (e.g. 2023.1). Versions in JIRA use the major version number, and append an integer as the patch identifier. The patch number starts with 0 for the GA release, and increments for each patch. SDP patches release on a schedule independent of P4D.
Develop and test changes in dev branch. This will occur over time.
When an issue is marked as complete, do the following in JIRA to get it included in the Release Notes:
P4Blog
field of the JIRA issue with the SDP review URL from The Workshop, e.g. https://swarm.workshop.perforce.com/reviews/29811Verify that the SDP regression test suite is successful.
The SDP regression test suite is presently only accessible from the Perforce corporate VPN.
Determine if the SDP release is to be Major, Patch, or Doc Only.
Starting with the 2021.2 release of P4D, SDP major releases are tied to the P4D release schedule, with SDP releases occurring when P4D releases, plus or minus no more than 10 business days.
Other releases will be Patch releases if there are any functionality changes, or Doc Only releases, where changes since the last release affect only documentation. The SDP documentation is key to its value, worth doing patch updates including only documentation changes.
This step applies to major releases only; skip this for patch releases.
Modify the script //guest/perforce_software/sdp/dev/helix_binaries/get_helix_binaries.sh
as follows:
Version=
value, incrementing the 3rd digit by one. So if the current value is 1.3.9
, increment to 1.3.10
.DefaultHelixVersion
value to match the current P4D GA release version with a value like r21.1
. The binaries for this release should be accessible http://ftp.perforce.com/perforce/r21.2 (replacing r21.2
with the new value).REMINDER: The DefaultHelixVersion
changes only for GA releases.
This step applies to major releases only; skip this for patch releases.
Update the SDP regression test suite to ensure that it references the new Helix Core release.
Here is a sample of the required change:
Change 29229 by tom_tyler@tom_tyler.P4MBPro4.test_sdp_dev on 2022/12/04 04:27:34
Accounted for change to default Helix Server version.
Affected files ...
... //guest/perforce_software/sdp/dev/Server/test/test_Upgrade.py#17 edit
Differences ...
==== //guest/perforce_software/sdp/dev/Server/test/test_Upgrade.py#17 (text+x) ====
95c95
< parser.add_argument('--to-version', default="22.1")
---
> parser.add_argument('--to-version', default="22.2")
This code change might possibly break the regression Verify that the SDP regression test suite is successful.
It is important to re-verify that the regression test suite is still successful after this change.
Update the doc/ReleaseNotes.adoc file, specifically the Change History section, by adding a new section titled something like SDP 2023.1 Patch 3. to include the JIRA issues fixed since the release or patch.
Run a JIRA report of SDP issues with the target Fix version, e.g. 2023.1.3, and ensure all issues listed in the report are listed in the release notes.
We don't need a JIRA issue for every change. Create JIRA issues to cover the essentials of what changed. Folks who want the gory details can look at changelists, but folks looking only at JIRA issues should get a useful summary of what's new in this release or patch, and if appropriate, what to be aware of when taking this release.
Load the latest SDP tools shell environment.
Change to the dir in your SDP workspace corresponding to //guest/perforce_software/sdp/tools
, and do p4 -s sync
followed by source ./env.sh
in that directory.
Ensure the dev
branch is updated from main.
This ensures any hot fixes that my have been made directly in main
branch are considered for incorporation in the formal release.
$ p4 merge -n -b perforce_software-sdp-dev
All revision(s) already integrated.
If there is anything to merge, take care of it in your regular SDP development workspace, not the release workspace. If needed, file new jobs and associate them with changes. Upon completion of any needed merge work.
This step applies to major releases only; skip this for patch releases.
Update AsciiDoc revnumber
and revdate
.
In all *.adoc files, verify that the :revnumber:
and :revdate:
tags in the file refer to the correct SDP release version and date the file was last modified -- updating it to the current date.
The fields look something like this example in each *.adoc file:
:revnumber: v2021.1
:revdate: 2021-10-20
Review documents for any outdated references or content.
This step applies to any release (major or patch) in which changes to the p4_vars.template
or instance_vars.template
files were made. Skip this for patch releases in which no changes to templates were made.
To verify whether changes are were made to the SDP Environment File since the last release or patch, do:
LastReleaseCL=$(p4 print -q //guest/perforce_software/sdp/main/Version|cut -d '/' -f 4|cut -d ' ' -f 1)
p4 changes //guest/perforce_software/sdp/dev/Server/Unix/p4/common/config/p4_vars.template@$LastReleaseCL,@now
p4 changes //guest/perforce_software/sdp/dev/Server/Unix/p4/common/config/instance_vars.template@$LastReleaseCL,@now
If no changes were made, skip to the next step. Otherwise, modify the script //guest/perforce_software/sdp/dev/Server/Unix/p4/common/sdp_upgrade/sdp_upgrade.sh
as follows:
Version=
value, incrementing the 3rd digit by one. So if the current value is 1.6.4
, increment to 1.6.5
.TAG:
comments.
SDPEnvFileNewFormatVersion
if the format of the SDP Environment File p4_vars
changed since the last release or patch. Check with SDPInstanceCfgFileNewFormatVersion
if the template for the Instance Config File changed since the last release or patch.SDPTargetMajorVersion
to the new version about to be released.Ensure generated documentation is current.
Run the gen_script_man_pages.sh -submit
and observe the output. It may or may not create a new changelist in //guest/perforce_software/sdp/dev/doc/gen/...
, depending on whether the underlying scripts have changed since they were last generated.
Generate HTML and PDF files from the AsciiDoc (*.adoc
) and Markdown files (*.md
).
Starting from the root of your SDP workspace, do:
cd dev/doc
make clean all
p4 rec
p4 -s submit -d "Regenerated docs." ...
Note: For the above to work, you will need some utilities to be installed, including asciidoc
, asciidoc-pdf
, and markdown
.
Push the files to main.
Change to the dev
branch directory in your workspace. Run mdcu.sh
(Merge Down/Copy Up) script, which takes no arguments and is not interactive. That script creates a pending changelist with the Copy Up main.
Massage the generated pending changelist description if desired.
WARNING FOR PARTIAL RELEASES: If this is to be a partial release, revert any files that are not ready to be released from the main branch. Also be sure the reverted files correlate to any jobs removed for a partial release.
Next, submit the generated pending changelist manually.
After running this script, run package_downloads.sh
. This puts tar and zip files into the downloads folder. Submit the generated changelist manually.
# SDP Release Process Overview ## Introduction This document describes the process used to release the Server Deployment Package (SDP). This is intended for staff developing the SDP. ## Preparation A dedicated SDP workspace should be used for releasing the SDP, separate from any you might use for SDP development. This dedicated release workspace should not contain any cruft files such as editor temp files. The release process mechanisms prevents known-cruft files from being added such as `*~` files and `.DS_Store` files on Mac. Using a clean dedicated release workspace guarantees no local excess files can sneak into the packaging process. ## Procedure ### Step 1: Plan! Use [this JIRA Project](https://perforce.atlassian.net/jira/software/c/projects/SDP/issues) to plan your work. This JIRA instance is only accessible with Perforce corporate SSO. The SDP product owner determines what the release will be. That is **2023.1.3** (aka **2023.1 Patch 3**). The SDP shares the same major version as Helix Core (e.g. 2023.1). Versions in JIRA use the major version number, and append an integer as the patch identifier. The patch number starts with 0 for the GA release, and increments for each patch. SDP patches release on a schedule independent of P4D. ### Step 2: Develop! Develop and test changes in dev branch. This will occur over time. When an issue is marked as complete, do the following in JIRA to get it included in the Release Notes: * Set the Fix version value to the next release, currently 2023.1.3. * Update the `P4Blog` field of the JIRA issue with the SDP review URL from The Workshop, e.g. https://swarm.workshop.perforce.com/reviews/29811 ### Step 3: Verify SDP Regression Tests Verify that the SDP regression test suite is successful. The [SDP regression test suite](https://workshop-jenkins.das.perforce.com:8080/jobs/sdp-dev) is presently only accessible from the Perforce corporate VPN. ### Step 4: Determine Release Type and Release Date Determine if the SDP release is to be Major, Patch, or Doc Only. Starting with the 2021.2 release of P4D, SDP major releases are tied to the P4D release schedule, **with SDP releases occurring when P4D releases, plus or minus no more than 10 business days**. Other releases will be Patch releases if there are any functionality changes, or Doc Only releases, where changes since the last release affect only documentation. The SDP documentation is key to its value, worth doing patch updates including only documentation changes. ### Step 5: Update Default P4D Version (Major Releases Only) This step applies to major releases only; skip this for patch releases. Modify the script `//guest/perforce_software/sdp/dev/helix_binaries/get_helix_binaries.sh` as follows: * Change the `Version=` value, incrementing the 3rd digit by one. So if the current value is `1.3.9`, increment to `1.3.10`. * Change the `DefaultHelixVersion` value to match the current P4D GA release version with a value like `r21.1`. The binaries for this release should be accessible http://ftp.perforce.com/perforce/r21.2 (replacing `r21.2` with the new value). **REMINDER**: The `DefaultHelixVersion` changes *only* for GA releases. ### Step 6: Target New P4D in SDP Regression Test Suite (Major Releases Only) This step applies to major releases only; skip this for patch releases. Update the SDP regression test suite to ensure that it references the new Helix Core release. Here is a sample of the required change: ``` Change 29229 by tom_tyler@tom_tyler.P4MBPro4.test_sdp_dev on 2022/12/04 04:27:34 Accounted for change to default Helix Server version. Affected files ... ... //guest/perforce_software/sdp/dev/Server/test/test_Upgrade.py#17 edit Differences ... ==== //guest/perforce_software/sdp/dev/Server/test/test_Upgrade.py#17 (text+x) ==== 95c95 < parser.add_argument('--to-version', default="22.1") --- > parser.add_argument('--to-version', default="22.2") ``` This code change might possibly break the regression Verify that the SDP regression test suite is successful. It is important to re-verify that the regression test suite is still successful after this change. ### Step 7: Update Release Notes Update the doc/ReleaseNotes.adoc file, specifically the **Change History** section, by adding a new section titled something like **SDP 2023.1 Patch 3**. to include the JIRA issues fixed since the release or patch. Run a JIRA report of SDP issues with the target Fix version, e.g. 2023.1.3, and ensure all issues listed in the report are listed in the release notes. We don't need a JIRA issue for every change. Create JIRA issues to cover the essentials of what changed. Folks who want the gory details can look at changelists, but folks looking only at JIRA issues should get a useful summary of what's new in this release or patch, and if appropriate, what to be aware of when taking this release. ### Step 8: Load Release Tools Load the latest SDP tools shell environment. Change to the dir in your SDP workspace corresponding to `//guest/perforce_software/sdp/tools`, and do `p4 -s sync` followed by `source ./env.sh` in that directory. ### Step 9: Merge Down Ensure the `dev` branch is updated from main. This ensures any hot fixes that my have been made directly in `main` branch are considered for incorporation in the formal release. ``` $ p4 merge -n -b perforce_software-sdp-dev All revision(s) already integrated. ``` If there is anything to merge, take care of it in your regular SDP development workspace, not the release workspace. If needed, file new jobs and associate them with changes. Upon completion of any needed merge work. ### Step 10: Update Doc Versions (Major Releases Only) This step applies to major releases only; skip this for patch releases. Update AsciiDoc `revnumber` and `revdate`. In all *.adoc files, verify that the `:revnumber:` and `:revdate:` tags in the file refer to the correct SDP release version and date the file was last modified -- updating it to the current date. The fields look something like this example in each *.adoc file: ``` :revnumber: v2021.1 :revdate: 2021-10-20 ``` Review documents for any outdated references or content. ### Step 11: Update TAG Refs in SDP Upgrade Script This step applies to any release (major or patch) in which changes to the `p4_vars.template` or `instance_vars.template` files were made. Skip this for patch releases in which no changes to templates were made. To verify whether changes are were made to the SDP Environment File since the last release or patch, do: ``` LastReleaseCL=$(p4 print -q //guest/perforce_software/sdp/main/Version|cut -d '/' -f 4|cut -d ' ' -f 1) p4 changes //guest/perforce_software/sdp/dev/Server/Unix/p4/common/config/p4_vars.template@$LastReleaseCL,@now p4 changes //guest/perforce_software/sdp/dev/Server/Unix/p4/common/config/instance_vars.template@$LastReleaseCL,@now ``` If no changes were made, skip to the next step. Otherwise, modify the script `//guest/perforce_software/sdp/dev/Server/Unix/p4/common/sdp_upgrade/sdp_upgrade.sh` as follows: * Change the `Version=` value, incrementing the 3rd digit by one. So if the current value is `1.6.4`, increment to `1.6.5`. * Search the file for `TAG:` comments. * Update `SDPEnvFileNewFormatVersion` if the format of the SDP Environment File `p4_vars` changed since the last release or patch. Check with * Update `SDPInstanceCfgFileNewFormatVersion` if the template for the Instance Config File changed since the last release or patch. * Update the `SDPTargetMajorVersion` to the new version about to be released. ### Step 12: Generate Script Docs Ensure generated documentation is current. Run the `gen_script_man_pages.sh -submit` and observe the output. It may or may not create a new changelist in `//guest/perforce_software/sdp/dev/doc/gen/...`, depending on whether the underlying scripts have changed since they were last generated. ### Step 13: Generate HTML and PDF Docs Generate HTML and PDF files from the AsciiDoc (`*.adoc`) and Markdown files (`*.md`). Starting from the root of your SDP workspace, do: ``` cd dev/doc make clean all p4 rec p4 -s submit -d "Regenerated docs." ... ``` Note: For the above to work, you will need some utilities to be installed, including `asciidoc`, `asciidoc-pdf`, and `markdown`. ### Step 14: Copy Up Push the files to main. Change to the `dev` branch directory in your workspace. Run `mdcu.sh` (Merge Down/Copy Up) script, which takes no arguments and is not interactive. That script creates a pending changelist with the Copy Up main. Massage the generated pending changelist description if desired. **WARNING FOR PARTIAL RELEASES**: If this is to be a partial release, revert any files that are not ready to be released from the main branch. Also be sure the reverted files correlate to any jobs removed for a partial release. Next, submit the generated pending changelist manually. ### Step 15: Package After running this script, run `package_downloads.sh`. This puts tar and zip files into the downloads folder. Submit the generated changelist manually.
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#24 | 30387 | C. Thomas Tyler | Updated SDP release process notes. | ||
#23 | 30358 | C. Thomas Tyler | Cleaned up notes about updating regression test suite for major versions. | ||
#22 | 30338 | C. Thomas Tyler |
Updated release process overview to account for changes needed to P4*.json release list files for each major release. |
||
#21 | 30293 | C. Thomas Tyler | Added new steps to udpate SDP Version file. | ||
#20 | 30111 | C. Thomas Tyler | Accounted for move of gen_script_man_pages.sh to SDP package tree. | ||
#19 | 30005 | C. Thomas Tyler | Refined slightly. | ||
#18 | 29835 | C. Thomas Tyler |
Updated Release Process to account for migration to JIRA. Removed scripts made obsolete by this change to avoid confusion. |
||
#17 | 29234 | C. Thomas Tyler |
Updated release process doc with note about updating test suite to use latest SDP for major releases, and added details. Fixed order bug. |
||
#16 | 29202 | C. Thomas Tyler | Fixed doc typo. | ||
#15 | 28653 | C. Thomas Tyler | Updated SDP release process docs. | ||
#14 | 28615 | C. Thomas Tyler | Documented new required steps in the SDP Relase Process. | ||
#13 | 28107 | C. Thomas Tyler | Replaced PRE tags with tripple-backticks for proper rendering. | ||
#12 | 28106 | C. Thomas Tyler | Text tweaks. | ||
#11 | 28105 | C. Thomas Tyler | Updated SDP Release Process. | ||
#10 | 27842 | C. Thomas Tyler | Minor updates to improve clarity of SDP release process overview doc. | ||
#9 | 27525 | C. Thomas Tyler | Tweaked notes to account for running on Mac, etc. | ||
#8 | 27465 | C. Thomas Tyler | Updated release notes. | ||
#7 | 27461 | C. Thomas Tyler | Refined AsciiDoc generated doc update part of release process. | ||
#6 | 27323 | C. Thomas Tyler | Tweaked release process doc. | ||
#5 | 27314 | C. Thomas Tyler | Refined release process overview notes. | ||
#4 | 27115 | C. Thomas Tyler |
Updated SDP release process doc, adding info on how to ensure generated script docs are incorporated in AsciiDoc files. Adjusted generated release notes file from .txt to .adoc format. Updated rel_notes.sh to add bulleted list items with links to each fixed job. Updated old release notes content to use new format (using disposable link conversion scripts). |
||
#3 | 24397 | C. Thomas Tyler | Tweaked SDP release process overview notes. | ||
#2 | 21524 | C. Thomas Tyler | Updated release process overview. | ||
#1 | 20479 | C. Thomas Tyler |
Added release process overview script. Overhauled release notes helper script. Fixed bug that listed a job in the release notes for the current release that was associated with the last changelist of the prior release. Updated more of the manual steps in the SDP release process. |