Change 33700
Make remove_old_checkpoints_and_journals()'s KEEPJNLS sweep offline-db-aware.
The .jnl.* retention sweep under $CHECKPOINTS previously had no idea whether
the offline database had actually replayed a journal before deleting it.
Confirmed live on eam-perforce1: this is what turned a transient offline-db
replay stall into a permanent, silent one -- once the sweep deleted a journal
number the offline db hadn't caught up to yet, replay could never advance
past it again (compounding with the two silent-failure bugs fixed in the
prior two changes, which is why it went unnoticed).
Now skips deleting any candidate journal number >= OFFLINEJNLNUM (extending
retention past KEEPJNLS instead), and sends a non-terminating WARNING
email/SNS + structured-log "warning" status when it does. Deliberately does
NOT reuse warn() here, since warn() always exit()s and this condition must
not abort the run (remove_old_logs/set_counter/ckp_complete still need to
run) -- mirrors journal_watch.sh's own existing non-terminating disk-space
warning pattern instead. Falls back to the prior unguarded behavior when
OFFLINEJNLNUM isn't set (several callers of this function never call
replay_journals_to_offline_db at all).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The .jnl.* retention sweep under $CHECKPOINTS previously had no idea whether
the offline database had actually replayed a journal before deleting it.
Confirmed live on eam-perforce1: this is what turned a transient offline-db
replay stall into a permanent, silent one -- once the sweep deleted a journal
number the offline db hadn't caught up to yet, replay could never advance
past it again (compounding with the two silent-failure bugs fixed in the
prior two changes, which is why it went unnoticed).
Now skips deleting any candidate journal number >= OFFLINEJNLNUM (extending
retention past KEEPJNLS instead), and sends a non-terminating WARNING
email/SNS + structured-log "warning" status when it does. Deliberately does
NOT reuse warn() here, since warn() always exit()s and this condition must
not abort the run (remove_old_logs/set_counter/ckp_complete still need to
run) -- mirrors journal_watch.sh's own existing non-terminating disk-space
warning pattern instead. Falls back to the prior unguarded behavior when
OFFLINEJNLNUM isn't set (several callers of this function never call
replay_journals_to_offline_db at all).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>