You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
tests/test_harvest.sh §7 ("Behavioural: harvest.sh --dry survives >20 commits") fails intermittently in ./tests/test.sh --unit with git plumbing errors that don't reproduce on a single-test re-run:
=== 7. Behavioural: harvest.sh --dry survives >20 commits ===
fatal: could not parse HEAD
or:
=== 7. Behavioural: harvest.sh --dry survives >20 commits ===
error: Could not read c8329c769513da7943f88d476e57ffbc63a02ebc
error: could not parse commit c8329c76...
fatal: the remote end hung up unexpectedly
Re-running ./tests/test.sh --unit (or ./tests/test_harvest.sh in isolation) passes 20/20. Five consecutive runs after a failure have all been clean.
Impact
Pre-merge gating: a CI run that hits this looks like a real regression when it isn't. Currently mitigated by re-running, but that erodes trust in the suite.
Likely root cause
§7 is by far the most I/O-heavy unit test: 100 --allow-empty commits with ~1 KiB subject padding, in a scratch repo under $TMPDIR, then git push to a scratch bare under /tmp.
Both error patterns are git's "I expected this ref/object to be readable and it wasn't" signature, which lines up with one of:
Auto-GC race. 100 loose objects approaches git's gc.auto threshold (default 6700 loose objects, but the trigger also depends on pack-refs timing). An auto-GC kicked off mid-§7 by an earlier test in the same suite could pack-then-prune under the test's feet.
Host config interference.test_harvest.sh does not export GIT_CONFIG_GLOBAL=/dev/null / GIT_CONFIG_SYSTEM=/dev/null (the isolation we added to test_launch.sh and test_session_end_push.sh in Pin Codex CLI, fix signing, isolate test gitconfig #91), so any host-side core.hooksPath, gc.*, maintenance.*, or pack.* config leaks in.
Filesystem hiccup on /tmp. Less actionable, but plausible on tmpfs/btrfs under load.
Proposed mitigations (not a fix yet — just lower flake rate)
If the flake survives both, the next step is GIT_TRACE=1 ./tests/test_harvest.sh on a failing run to see what git was actually doing when it lost the HEAD ref.
Repro
Not consistently reproducible. Observed during runs of ./tests/test.sh --unit on the master branch and on fix/codex-at-capacity-retriable (PR for #85), with two distinct error wordings on consecutive failures. Five clean re-runs in a row immediately after both failures.
Version
0.20.12 (and 0.20.13-pending on the codex-at-capacity branch).
Summary
tests/test_harvest.sh§7 ("Behavioural: harvest.sh --dry survives >20 commits") fails intermittently in./tests/test.sh --unitwith git plumbing errors that don't reproduce on a single-test re-run:or:
Re-running
./tests/test.sh --unit(or./tests/test_harvest.shin isolation) passes 20/20. Five consecutive runs after a failure have all been clean.Impact
Pre-merge gating: a CI run that hits this looks like a real regression when it isn't. Currently mitigated by re-running, but that erodes trust in the suite.
Likely root cause
§7 is by far the most I/O-heavy unit test: 100
--allow-emptycommits with ~1 KiB subject padding, in a scratch repo under$TMPDIR, thengit pushto a scratch bare under/tmp.Both error patterns are git's "I expected this ref/object to be readable and it wasn't" signature, which lines up with one of:
gc.autothreshold (default 6700 loose objects, but the trigger also depends on pack-refs timing). An auto-GC kicked off mid-§7 by an earlier test in the same suite could pack-then-prune under the test's feet.test_harvest.shdoes not exportGIT_CONFIG_GLOBAL=/dev/null/GIT_CONFIG_SYSTEM=/dev/null(the isolation we added totest_launch.shandtest_session_end_push.shin Pin Codex CLI, fix signing, isolate test gitconfig #91), so any host-sidecore.hooksPath,gc.*,maintenance.*, orpack.*config leaks in./tmp. Less actionable, but plausible on tmpfs/btrfs under load.Proposed mitigations (not a fix yet — just lower flake rate)
Two cheap, additive changes worth trying first:
and inside §7's scratch repos:
If the flake survives both, the next step is
GIT_TRACE=1 ./tests/test_harvest.shon a failing run to see what git was actually doing when it lost the HEAD ref.Repro
Not consistently reproducible. Observed during runs of
./tests/test.sh --uniton the master branch and onfix/codex-at-capacity-retriable(PR for #85), with two distinct error wordings on consecutive failures. Five clean re-runs in a row immediately after both failures.Version
0.20.12 (and 0.20.13-pending on the codex-at-capacity branch).