= SDP Windows to Linux Migration Guide Perforce Professional Services <consulting@perforce.com> :revnumber: v2023.1 :revdate: 2023-11-29 :doctype: book :icons: font :toc: :toclevels: 5 :sectnumlevels: 4 :xrefstyle: full // Attribute for ifdef usage :unix_doc: true == Preface This guide documents the process for migrating a Helix Core service from Windows server to Linux. A migration can be minimally disruptive to users if planned and executed properly, and has significant benefits. The purpose of this document is to help inform the planning. For purposes of this document, it does not matter if the servers are on-premises ("on-prem") or in a private or public cloud environment such as AWS, Azure, or GCP. The Windows service may or may not be operated using Windows SDP. Regardless of whether the Windows service is managed with SDP, the Windows service is left alone during the migration. *Please Give Us Feedback* Perforce welcomes feedback from our users. Please send any suggestions for improving this document to consulting@perforce.com. :sectnums: == Overview A Migration has these elements: * Planning: See <<_migration_planning>>. * Upgrade P4D to 2019.1+ on Windows. * Provision New Linux Server machine. * Install Perforce Helix software on Linux. * Correct data issues identified in planning. * Setup Linux Replica server spec on Windows. * Pull archives. This may take a long while if there is a lot of data to pull. Each of these components is covered, in detail, in this guide. == Migration Planning There are several things to account for in planning a Windows to Linux migration. === Migration Strategy Several migration strategies are possible. This document focuses on the Failover Style strategy. This strategy has several benefits: * Minimum disruption to end users for the cutover. * Allows for extensive testing of the new Linux server(s) and infrastructure prior to cutover. * The affect on the original Windows server(s) and infrastructure is minimal. * Rollback, while hopefully not necessary, is straightforward. While planning and preparation will take time and effort, the disruption to end users can be minimal. === Incompatible Configuration Settings Using the `p4 configure` command to interact with `db.config` is a good way, and in many cases the only way, to set various configuration items with a Helix Core server. However, there are certain settings that must not be defined with `p4 configure`, as they conflict with settings the SDP defines. Review the output of `p4 configure show allservers` and see if any of the following are set: * `P4JOURNAL` * `P4PORT` * `P4LOG` If any of these are set with `p4 configure`, the migration plan will need to deal with unsetting them. === Depot Root and Depot Spec Map Fields Perforce Helix depot specs have a field named `Map:` that, if used, must be eliminated prior to the deployment of a Linux replica. Further, the `server.depot.root` configurable must be set on the commit server. If done carefully, the changes to set `server.depot.root` and clear the `Map:` field of each depot spec can be done non-disruptively on the live running Windows Perforce Helix Core service, and must be done before creating the checkpoint used to seed the Linux replica. The key to making the change non-disruptively is to understand that the p4d server will use the `Map:` field value to see if it is set, Essentially, you need to eliminate them. If you're using multiple drives for depots or Windows junctions or UNC paths or some such, you need to make directory symlinks to point to those things in a single depot, so that you can set aserver.depot.root value that works on Windows for all depots. You'll need to change server.depot.root and journalPrefix on the ServerID of the Linux Replica. Think "Big Blue/Green Deploy" - every server in the topology, replicas, edges, etc. all need to go Windows -> Linux. Replicas can just be rebuilt; handling edges is more complex but doable. Don't for proxies -- they need to do the Windows->Linux thing too. (They could have been Linux all along even with a Windows P4D, but don't forget to check that). Any custom triggers will need to be reviewed -- any that can't be discarded will need to evaluated for porting and testing needs. Plan to 3 cycles of p4verify.sh, to get p4d to pull the archives. The first, starting with no archive files, is to start a bulk pull. That could take days or weeks depending on data scale. The second to fill in gaps, and the 3rd should be clean. Depending on scale of data, you may want to consider using outside-p4d mechanisms for transferring some archives (especially gz files, ,v files should be p4 pull'd ideally). Lots of variations on how to get the archives files there. Using p4 pull is better in that, if the Linux p4d writes the archive, it can always find it, even it it's funky with Unicode cruft in the path, whereas things copied outside p4d may not be found by the Linux p4d. However, for bulk pulls of Terabytes of data, a Windows port of rsync, at least for .gz files, will be faster. You'll probably need a live running rsync service on Linux for the Windows port of rsync to talk to. There are many options here; somehow or other get the files in place so p4verify.sh is happy. There are some temporary custom hacks you can put into the Linux SDP to allow you to run the mkrep.sh script against the Windows master to get things started. Normally in an Linux server world, you run mkrep.sh on the commit server. But when your commit server is Windows, you run the mkrep.sh on the Linux replica-to-be machine, after SDP is installed but before you have any data brought over. It creates the server spec, service user, and all that stuff that you need backed into a checkpoint. Note: If folks want case conversion as well (often just going to Linux is enough), I strongly advise doing a 2-step migration. Step 1 is go from Windows to Linux, case-insensitive. Then once you're live and on Linux, do the case conversion as a separate project. Important: For this to be done with a failover, the Linux p4d should be 2019.1+. (p4 failover came in 2018.2, but you want p4d start point to be 2019.1+ latest patch). Typically we do the failover-then-upgrade in the same maintenance Windows. That is, failover to the new server on Linux on the same p4d server version as Windows. Then once on Linux, do the standard SDP upgrade procedure for Linux, using upgrade.sh. === Deploy New Linux Server machine ==== Select Operating System As of this writing, the best options are: * Ubuntu 20.04 (not 22.04 just yet) * RHEL/Rocky Linux 8 (not 9 just yet) * Amazon Linux 2 (not Amazon Linux 2023 just yet) ==== Install Helix Core Software Use the Helix Installer, do a Configured Install: su - mkdir -p /hxdepots/reset cd /hxdepots/reset curl -L -s -O https://swarm.workshop.perforce.com/download/guest/perforce_software/helix-installer/main/src/reset_sdp.sh chmod +x reset_sdp.sh ./reset_sdp.sh -C > settings.cfg In `settings.cfg`, change these settings: * DNS_name_of_master_server= * SiteTag= * P4_PORT= * Instance= * Password= * CaseSensitive= * P4USER= * ServerID= * ServerType= * P4BinRel= * P4APIRel= Then run the script: ./reset_sdp.sh -no_sd -c settings.cfg 2>&1 | tee log.reset_sdp.txt su - perforce p4 set cd /p4/common/site [[ -d config ]] || mkdir config cd config Temporary Hack: vi /p4/common/site/config/p4_N.vars.local export P4MASTER_ID=Master export P4MASTERPORT=120.2:43430 export P4PORT=$P4MASTERPORT vi /p4/common/config/.p4passwd.p4_mocap.admin p4login -v cd /p4/common/config vi SiteTags.cfg azwestus2: Azure data center Add to Protections: super group ServiceUsers * //... mkrep.sh -t fs -s azwestus2 -r TestMachine Undo Temporary Hack: vi /p4/common/site/config/p4_N.vars.local #export P4MASTER_ID=Master #export P4PORT=$P4MASTERPORT export P4MASTERPORT=120.2:43430 [appendix] == Why Migrate? Migrations from Windows to Linux have been the single most consistent theme in Perforce Consulting in many years, for many reasons.
# | Change | User | Description | Committed | |
---|---|---|---|---|---|
#17 | 30383 | C. Thomas Tyler | Updated rev{number,date} fields in adoc files for release. | ||
#16 | 30342 | C. Thomas Tyler |
Fixed bogus revs to the dev branch in released docs. #review-30343 @amo |
||
#15 | 30207 | C. Thomas Tyler | Doc work WIP | ||
#14 | 30206 | C. Thomas Tyler | WIP. | ||
#13 | 30204 | C. Thomas Tyler | Doc work in progress ... | ||
#12 | 30202 | C. Thomas Tyler | Doc work in progress ... | ||
#11 | 30201 | C. Thomas Tyler | Doc work in progress ... | ||
#10 | 30178 | C. Thomas Tyler | Doc in progress updates. | ||
#9 | 30159 | C. Thomas Tyler | Documentation work in progress. | ||
#8 | 30000 | C. Thomas Tyler |
Refined Release Notes and top-level README.md file in preparation for coming 2023.2 release. Adjusted Makefile in doc directory to also generate top-level README.html from top-level README.md file so that the HTML file is reliably updated in the SDP release process. Updated :revnumber: and :revdate: docs in AsciiDoc files to indicate that the are still current. Avoiding regen of ReleaseNotes.pdf binary file since that will need at least one more update before shipping SDP 2023.2. |
||
#7 | 29953 | C. Thomas Tyler | Regeneratd docs. | ||
#6 | 29937 | C. Thomas Tyler | Very rough draft still WIP. | ||
#5 | 29934 | C. Thomas Tyler | WIP. | ||
#4 | 29931 | C. Thomas Tyler | WIP - Windows to Linux guide; still very raw. | ||
#3 | 29742 | C. Thomas Tyler | WIP. | ||
#2 | 29741 | C. Thomas Tyler | WIP. | ||
#1 | 29740 | C. Thomas Tyler | Bootstrapped empty doc. |