env.sh #5

  • //
  • guest/
  • perforce_software/
  • sdp/
  • dev/
  • test/
  • env.sh
  • View
  • Commits
  • Open Download .zip Download (4 KB)
#------------------------------------------------------------------------------
# Convenience aliases for testing.
#------------------------------------------------------------------------------

#------------------------------------------------------------------------------
# Usage:
# cd /pub/sdp/dev/test
# . ./env.sh
#------------------------------------------------------------------------------

declare _SDP_TEST_HOME=$PWD

#------------------------------------------------------------------------------
# Sync Latest SDP Workspace, and optionally unshelve some changeslists.
#
# This assumes a test SDP workspace has a .p4config file containig a setting
# like:
# P4CLIENT=tom_tyler.P4MBPro3.test_sdp_dev
#
# It also assumes P4PORT and P4USER context settings are already defined
# appropriately for accessing the Public Depot, perhaps with a higher level
# .p4config file.
#
# Usage:
#   sl
#   sl 25101 25302
#
# Each call reverts any opened files in the workspace, syncs to the tip, and
# unselves any listed shelved changelists.  She shelves are expected to be
# in the same workspace as the workspace defined by $_SDP_TEST_HOME, typically
# 'dev' or 'main'.#
#
# ***WARNING***: This worfklow assumes you're not doing any actual editing or
# work in the test workspace, but only using it to get files into the Docker
# test suite for testing there.
#------------------------------------------------------------------------------
function sl () {
   declare p4configFile="$_SDP_TEST_HOME/sdp/.p4config"
   declare changelistsToUnshelve=
   declare cmd=
   declare allOK=1
   [[ $# -ge 0 ]] && changelistsToUnshelve="$*"

   user=$(p4 set -q P4USER)
   if [[ -r "$p4configFile" ]]; then
      ws=$(grep P4CLIENT= sdp/.p4config)
      ws=${ws#*=}
      if [[ -n "$ws" ]]; then
         cmd="p4 -c $ws -s revert //$ws/..."
         echo "Running: $cmd"
         if ! $cmd; then
            echo -e "\\nError: Revert failed, not syncing.\\n"
            return 1
         fi
         cmd="p4 -c $ws -s sync -q --parallel=threads=8"
         echo "Running: $cmd"
         if ! $cmd; then
            echo -e "\\nError: Sync failed, not unshelving.\\n"
            return 1
         fi
         if [[ -n "$changelistsToUnshelve" ]]; then
            for cl in $changelistsToUnshelve; do
               cmd="p4 -c $ws unshelve -s $cl"
               echo "Running: $cmd"
               if ! $cmd; then
                  echo -e "\\nError: unshelve failed for $cl, skipping it.\\n"
                  allOK=0
               fi
            done
            if [[ "$allOK" -eq 1 ]]; then
               echo -e "\\nUnshelving was successful for changes $changelistsToUnshelve."
            else
               echo -e "\\nError: unshelve failed for one or more changelists.\\n"
               allOK=0
            fi
         else
            echo "No shelved changelists specified to unshelve."
         fi
      else
         echo -e "\\nError: Could not determine P4CLIENT value from $p4configFile. Not syncing."
         return 1
      fi
   else
      echo -e "\\nError: Missing $p4configFile.  To use the 'sl' (sync latest)\\nfunction, create a file $p4configFile containing:\\n\\nP4CLIENT=<your_sdp_test_workspace>\\n\\nSee: $p4configFile/$README.md\\n"
      return 1
   fi
}

# Run Docker Tests for CentOS 6
alias dr6='docker run --rm -e TESTOS=centos6 -v $PWD/sdp:/sdp perforce/centos6-sdp /p4/docker_entry.sh'

# Docker Interative Session for CentOS 6
alias di6='docker run --rm -e TESTOS=cenots6 -v $PWD/sdp:/sdp -it perforce/centos6-sdp /bin/bash'

# Run Docker Tests for CentOS 7
alias dr7='docker run --rm -e TESTOS=centos7 -v $PWD/sdp:/sdp perforce/centos7-sdp /p4/docker_entry.sh'

# Docker Interative Session for CentOS 7
alias di7='docker run --rm -e TESTOS=centos7 -v $PWD/sdp:/sdp -it perforce/centos7-sdp /bin/bash'

# Run Docker Tests for Ubuntu
alias dru='docker run --rm -e TESTOS=ubuntu20 -v $PWD/sdp:/sdp perforce/ubuntu20-sdp /p4/docker_entry.sh'

# Docker Interative Session for Ubuntu
alias diu='docker run --rm -e TESTOS=ubuntu20 -v $PWD/sdp:/sdp -it perforce/ubuntu20-sdp /bin/bash'
# Change User Description Committed
#8 30356 C. Thomas Tyler Added Rocky 9 and Ubuntu 22 to test suite.

Dropped CentOS 6.

Changed default OS for tests from CentOS 7 to Rocky 9.

Adapted to using podman for the test suite.

Reduced minimum disk space requirement for testing
from 1G to 10M.

To be revisited:
* Disabled OOM killer defenese feature and systemd due to
compatibility issues with the test environment.
* Disabled systemd by moving systemctl aside

Enhanced build_docker_image.sh; added '-clean' option and added
support for more platforms.

Updated OS-specific packages as needed for all platforms, e.g.
adding 'file' and 'rsync' utils as needed in Docker definitions.

Added minimum performance recommendations for test containers in
config.

Updated env.sh utilities.

This change test on:
[X] centos7
[X] rocky8
[X] rocky9
[X] ubuntu20
[X] ubuntu22

#review-30357 @robert_cowham

TO DO:
* This builds using the official SDP test suite run manually on
Tom's laptop. Next step: Get it working on our shiny new Jenkins
Rocky 9 server machine.
#7 28850 C. Thomas Tyler Test suite tweaks:
* Added command line usage notes in scripts.
* Enhanced comand line processing.
* Added aliases for building on Rocky Linux 8.

#review-28851
#6 27722 C. Thomas Tyler Refinements to @27712:
* Resolved one out-of-date file (verify_sdp.sh).
* Added missing adoc file for which HTML file had a change (WorkflowEnforcementTriggers.adoc).
* Updated revdate/revnumber in *.adoc files.
* Additional content updates in Server/Unix/p4/common/etc/cron.d/ReadMe.md.
* Bumped version numbers on scripts with Version= def'n.
* Generated HTML, PDF, and doc/gen files:
  - Most HTML and all PDF are generated using Makefiles that call an AsciiDoc utility.
  - HTML for Perl scripts is generated with pod2html.
  - doc/gen/*.man.txt files are generated with .../tools/gen_script_man_pages.sh.

#review-27712
#5 27002 C. Thomas Tyler Updated ubuntu tests to add needed utilities, and change from
testing Ubuntu 16.04 to 20.04.  Changed 'centos' to imply cento7
rather than centos6 (which we shoud drop soon).
#4 27001 C. Thomas Tyler Added convenience aliases for testing.
#3 26982 C. Thomas Tyler mkdirs.sh v4.1.0:
* Accounted for directory structure change of Maintenance to Unsupported.
* Added standard command line processing with '-h' and '-man' doc flags,
and other flags (all documented).
* Added in-code docs and updated AsciiDoc.
* Enhanced '-test' mode to simulate /hx* mounts.
* Enhanced preflight testing, and fixed '-test' mode installs.
* Added support for installing to an alternate root directory.
* Added '-s <ServerID>' option to override REPLICA_ID.
* Added '-S <TargetServerID>' used for replicas of edge servers.
* Added '-t <server_type>' option to override SERVER_TYPE.
* Added '-M' option to override mount points.
* Added '-f' fast option to skip big chown/chmod commands, and
moved those commands near the end as well.

verify_sdp.sh v5.9.0:
* Added check for /p4/Version file, and checked that other legacy
SDP methods of checking version
* Added sanity check for crontab.
* Added 'test skip' mechanism to skip certain tests:
 - crontab: Skip crontab check. Use this if you do not expect crontab to be configured, perhaps if a different scheduler is used.
 - license: Skip license related checks.
 - version: Skip version checks.
 - excess: Skip checks for excess copies of p4d/p4p/p4broker in PATH.
* Added VERIFY_SDP_SKIP_TEST_LIST setting ton instance_vars.template,
to define a standard way to have verify_sdp.sh always skip certain
tests for a site.
* Extended '-online' checks to check for bogus P4MASTERPORT, a common
config error.

Update test_SDP.py:
* Adjusted test suite to account for various changes in mkdirs.sh.
* Added 'dir' parameter to run_cmd() and sudo_cmd(), to run a
command from a specified directory (as required to test new
mkdirs.sh)
* Added check_links() similar to existing check_dirs() function.

=== Upgrade Process Changes ===

Made /p4/common/bin/p4d/p4/p4broker/p4p shell script rather than binary.

This changes the way SDP new binaries are staged for upgrade.  For
safety, exes are now staged to a director outside the PATH, the
/p4/sdp/exes folder. A new 'get_latest_exes.sh' script simplifies
the task of pulling executables from the Perforce FTP server. This
can be used 'as is' for environments with outbound internet access,
and is useful in any case to describe now to acquire binaries.

This addresses an issue where a p4d binary staged for a future
upgrade might be called before the actual upgrade is performed.

upgrade.sh v4.0.0:
* All preflight checks are now done first. Added '-p' to abort after preflight.
* Added '-n' to show what would be done before anything is executed.
* Minimalist logic to start/stop only servers that are upgrade, and apply
upgrades only as needed.
* Staging of exes for upgrade is now separate from /p4/common/bin
* Improved in-code docs, added '-h' and '-man' options.
* Retained pre/post P4D 2019.1 upgrade logic.
#2 25474 C. Thomas Tyler Added support for an unshelving workflow with the 'sl' function
in the test SDP workspace on which the Docker test suite runs.
#1 25434 C. Thomas Tyler Added convenience aliases and function to provide shorthands for
Docker test environment workflow.