# sed_fixer.sh Development and Backslash-Style Cleanup — Session Log

**Date:** 2026-09-05
**Workspace:** `/Users/ttyler/pub/p4-hms` (client `tom_tyler.P4MBPro4.p4-hms`, stream `//p4-hms/dev`)
**Result:** Change 33528 (add `tools/sed_fixer.sh`), Change 33529 (20 files, cosmetic `\\n`/`\\t` → `\n`/`\t` normalization)

## Summary

Debugged and extended a new, not-yet-working script (`tools/sed_fixer.sh`)
that bulk-applies grep-selected sed substitutions across a file tree with
Perforce checkout integration, then used it to perform a real, tree-wide
cosmetic cleanup.

## Phases

1. **Fixed the core bug**
   - `SedExpressionList` elements were being expanded raw
     (`"${SedExpressionList[@]}"`) into the `sed -E -i` command with no `-e`
     in front of each one, so sed treated the first expression as its
     script and the rest as filenames. Fixed by building a companion
     `SedArgs` array (`-e expr1 -e expr2 ...`) once after argument parsing,
     used by both the GNU-sed and BSD/macOS-sed branches.

2. **Added options and docs**
   - `-dir <dir>`: overrides the default operating directory (previously
     hardcoded to one level above the invocation directory); validated to
     exist.
   - `-nop4`: disables all Perforce interactions (ticket check, opened-file
     scan, `p4 edit`); auto-implied with a warning if `p4 info` doesn't
     respond.
   - Wrote full `DESCRIPTION`/`OPTIONS`/`FILES`/`EXAMPLES` sections for
     `usage -man`, following the SDP script template style
     (`~/pub/sdp/dev/Server/Unix/p4/common/bin/templates/template.sh`).

3. **Found and fixed a second real bug**
   - The main loop checked `[[ -w "$File" ]]` *before* attempting
     `p4 edit`. Normal p4-controlled files are read-only until checked
     out, so this ordering meant the script would silently skip every
     real file without ever attempting checkout (0 errors reported, but
     nothing done). Reordered: checkout (or the `-nop4` path) now happens
     first, and writability is verified afterward, before running sed.
     This was only caught by testing against an actual centralized-server
     file, not the initial scratch-directory tests.

4. **Testing**
   - Isolated scratch-directory tests (`-nop4`) for the sed-array fix
     itself.
   - Real centralized-server test: checked out `tools/components.txt`,
     confirmed the fix worked, then `p4 revert`'d it — no permanent depot
     change.
   - Disposable DVCS micro-repo test (`p4 init -C0` under `/tmp`, no
     server process, throwaway ticket/password set on the local user):
     multi-file checkout + single-pass multi-`-e` sed, `-co`-vs-default
     already-opened-file abort behavior, and cleanup via revert. Exposed
     one DVCS-only artifact (grep wandering into the co-located
     `.p4root` server database and hitting a binary file) — not a script
     bug, just a DVCS quirk; noted but not changed.
   - Committed `tools/sed_fixer.sh` as **change 33528**.

5. **Real usage: `\\n`/`\\t` → `\n`/`\t` cosmetic cleanup**
   - Initial ask was ambiguous between two very different operations:
     (a) converting literal two-character `\n`/`\t` text into actual
     embedded newline/tab bytes, vs. (b) shortening literal `\\n`/`\\t`
     (double backslash) to `\n`/`\t` (single backslash) as plain text.
   - Investigated (a) first: found 39 matching files across the tree, but
     spot-checking context showed most matches were legitimate working
     Perl/Python/bash escape sequences (e.g. `die("...$!\n")` in
     `JobIncrement.pl`, `'\n'.join(out)` in `hms_mindmap.py`) — inserting
     a raw newline byte there would have broken Python syntax outright
     and was flagged as a risk for a live broker config template
     (`p4_N.broker.cfg.t`) whose parser tolerance was unknown. Paused and
     asked before proceeding.
   - User clarified the actual intent: (b) — a pre-2026 ShellCheck version
     once recommended double-backslash-escaping `\n`/`\t` in double-quoted
     bash strings; a later ShellCheck version reversed that guidance.
     Both forms are functionally identical at runtime; the double-backslash
     form is just visual clutter the user wanted normalized.
   - Re-derived and verified the correct grep/sed expressions on
     controlled test files before touching anything real:
     `GrepExpression='\\\\[nt]'`, `SedExpression1='s@\\\\n@\\n@g'`,
     `SedExpression2='s@\\\\t@\\t@g'` — confirmed these convert only
     literal double-backslash sequences to single-backslash, leaving
     already-correct single-backslash text and real embedded tab/newline
     bytes untouched.
   - Re-scanned the real tree with the corrected pattern: 20 files, all
     bash (`.sh` or shebang-confirmed `#!/bin/bash`), consistent with a
     bash-linter-driven origin and no cross-language risk.
   - Ran `tools/sed_fixer.sh` live (default directory, one level above
     `tools/`); all 20 files processed with 0 errors.
   - Spot-checked `p4 diff` on 4 files across different subsystems
     (`pre-sdp_upgrade.sh`, `bin/hms`, `gjh.sh`,
     `dlp/test/run_tests.sh`) — every change was exactly `\\n`→`\n` /
     `\\t`→`\t`, nothing else touched.
   - Ran `bash -n` across all 20 changed files as a final safety net — all
     passed.
   - Submitted as **change 33529**.

## Process note

Per the existing convention in this folder (see
`2026-09-04-consistency-pass.md`), this log is being submitted as its own
changelist, separate from the `sed_fixer.sh` and cosmetic-cleanup changes
it describes (33528, 33529).