# Session Log: First Real Dry Run -- Found and Fixed a Silent Data-Loss Bug **Date:** 2026-08-31 **Author:** Claude (Claude Code session) **Requested by:** Tom Tyler --- ## Context Continuation of the SDP-coding-standard alignment work (change 33383). Tonight's ask: actually run `sync_jira_to_p4jobs.sh` end-to-end for the first time -- dry run (no `-y`), `-max 5` -- against real JIRA and P4, and report where the log lands. ## What it took to get a real run going 1. **P4 ticket file.** The script needs `P4PasswdFile` (a flat file with just the ticket, from `p4 login -p > file`). Writing that file was blocked twice by the Claude Code auto-mode safety classifier as credential extraction. Tom ran the command himself: `p4 login -p > /Users/ttyler/pub/j2j/secure/p4_ticket.value && chmod 600 ...` Tom noted a dedicated service account should replace his personal ticket/token before any live (`-y`) run -- personal credentials are fine for tonight's dry run only. **Follow-up, not done yet.** 2. **Test config.** Wrote `/Users/ttyler/pub/j2j/secure/dryrun_test.cfg` (inside the already-`.p4ignore`d `secure/` tree, `chmod 600`): real `JiraUrl`/`JiraUser` (`ttyler@perforce.com`, since JIRA Cloud Basic auth needs the account email, not the `ttyler` value in `token.user`), the copied `token.value`, `JiraProjects=SDP`, and `P4PORT=public.perforce.com:1666` / `P4USER=tom_tyler` (the same server/account already used to version this very script -- confirmed working via `p4 info` in an earlier session, rather than the untested `ssl:workshop.perforce.com:1666` from the README example). 3. **`flock` missing on macOS.** First run failed immediately: `flock: command not found`. `flock` is a Linux `util-linux` tool, already listed as a prerequisite in the README, but not present on macOS by default. Installed a compatible reimplementation via `brew install flock` (discoteq/flock 0.4.0) -- unblocked local testing without needing to move to a Linux box, which Tom had offered as a fallback if needed. ## First run result: looked clean, was actually broken With those two things in place, `-C secure/dryrun_test.cfg -max 5 -v 5` ran to completion with `Errors=0 Warnings=0` and reported **0 issues fetched**. That looked like a legitimate "nothing changed in this window" result, but it wasn't -- a direct `curl` call using the exact same JQL and URL the script had just printed returned 50 real issues. **Root cause:** `pageJson=$(jira_get "$url")` captures `jira_get()`'s stdout as its return value. But `jira_get()` also calls `detail()` (verbosity 4+) and `warnmsg()` (always, on any retry) -- and both of those wrote to **stdout**, the same stream being captured. At `-v 5`, the `detail " JIRA GET (attempt 1/3): ..."` line landed in front of the actual JSON inside `$pageJson`, so `json.loads()` choked on it. That failure was silently swallowed by the script's own `2>/dev/null || echo 0` fallback (used throughout to avoid a jq dependency), which is indistinguishable from a legitimately empty page -- so the run reported a clean 0-issue success and the watermark advanced right past 50 real, unsynced issues. Confirmed by inspecting the raw cached page file (`secure/state_test/jira_page_cache/SDP_page1.json`), which had the debug line prepended to the real 170KB JSON payload. **Why this matters beyond `-v 5`:** `warnmsg()` isn't gated by verbosity -- it always prints. So this wasn't just a debug-mode artifact: any transient JIRA rate-limit (429) or 5xx retry, at any verbosity, including the default production setting, would corrupt that page's captured JSON the same way, get treated as "0 issues, no error," and silently advance the watermark past real issues -- permanent, silent data loss with no error logged. `map_jira_project_to_p4()` had the identical exposure via its own `warnmsg` call, corrupting the returned P4 `Project` value instead. **Fix (change 33384):** all diagnostic output (`msg`/`msgn`/`dbg`/`detail`, and transitively `warnmsg`/`errmsg`) now writes to stderr, never stdout. Stdout is reserved exclusively for actual function return values. No visible-output change, since `init_logging()` already merges stderr into the same tee'd/logged stream via `2>&1`. **Re-ran after the fix:** fetched 50 issues, synced 5 (respecting `-max 5`), correctly stopped early and logged "Watermark NOT advanced (-max 5 reached)," fields mapped as expected (e.g. `SDP-1172 status=open type=Problem project=perforce-software-sdp`). ## Log location Confirmed the log path convention works as designed: - `/tmp/sync_jira_to_p4jobs..log` (this run: `/tmp/sync_jira_to_p4jobs.2026-08-31-231130.log`) - `/tmp/sync_jira_to_p4jobs.log` -- symlink to the above, byte-identical to the timestamped file (diffed to confirm). - Lands in `/tmp` because `$LOGS` isn't set in this ad hoc local environment (no SDP `p4_vars` sourced); on a real SDP host it'll be `$LOGS` (`/p4//logs`) per the documented convention. ## Minor observation, not fixed tonight `jira_type_to_p4()` mapped all 5 fetched issues to `Problem` -- they were all JIRA type "Feature Request", which doesn't match the `"new feature"*` pattern in the case statement (JIRA's actual type name doesn't start with "new"). Falls through to the `Problem` default, which is a safe fallback, not a crash, but likely not the intended mapping for a fairly common SDP issue type. Worth adding an explicit `("feature request")` arm alongside `story`/`epic`/`task` next time this file is touched. ## Follow-ups (not done tonight) 1. **Dedicated JIRA + P4 service accounts** before any live (`-y`) run -- Tom flagged this himself. Currently using his personal API token and P4 ticket, fine for dry runs only. 2. **`jira_type_to_p4()` "Feature Request" mapping** (above). 3. Everything already listed in `SESSION_2026-08-30_dry_run_and_next_steps.md` remains open: the `curl --user` argv exposure (same class of issue as the P4 `-P` fix, not yet addressed) and the `p4 fix` / `P4Blog` idea.