tasks/github/standard/mcpmark-cicd/linting_ci_workflow/verify.py:288 waits for
GitHub Actions runs to complete:
start_time = time.time()
timeout = 300
while time.time() - start_time < timeout:
...
time.sleep(10)
while src/base/task_manager.py:261 runs that verifier as
subprocess.run(self._get_verification_command(task), capture_output=True,
text=True, timeout=300)
The two budgets are equal, so once the poll loop actually needs its full 300 s the
subprocess is killed before the loop exits — the checks after the loop, which are
the ones that produce the verdict, never run. The result is a TimeoutExpired
recorded as a verification error instead of a pass or a fail.
This looks like an unintended consequence of #255, which raised this verifier's
workflow-completion wait from 90 s to 300 s ("for cold GH runners + npm ci + glob").
The change reached main in the #264 release, and the diff on this file is exactly
- timeout = 90 / + timeout = 300. At 90 s the loop always finished well
inside the subprocess budget; at 300 s it exactly meets it, so the headroom is gone.
Related but not a fix: open PR #223 adds an explicit except subprocess.TimeoutExpired
around this call, reporting "Verification script timed out after 300s". That makes the
failure legible, but it keeps timeout=300, so the run still ends without a verdict.
It only bites when the awaited CI state does not settle quickly: in our runs 22 of
25 verified in 5–43 s, and 3 stalled and were killed at ~305 s, losing their grade.
Neither number is configurable — the 300 in run_verification is a literal,
repeated across six task managers, and github uses the base class's.
Two ways to close it, either is fine:
- lower the verifier's own poll budget so it always fits (e.g. 240 s), or
- raise
run_verification's timeout above the longest verifier poll, ideally
deriving one from the other rather than repeating a constant.
Happy to send a PR for whichever you prefer.
tasks/github/standard/mcpmark-cicd/linting_ci_workflow/verify.py:288waits forGitHub Actions runs to complete:
while
src/base/task_manager.py:261runs that verifier asThe two budgets are equal, so once the poll loop actually needs its full 300 s the
subprocess is killed before the loop exits — the checks after the loop, which are
the ones that produce the verdict, never run. The result is a
TimeoutExpiredrecorded as a verification error instead of a pass or a fail.
This looks like an unintended consequence of #255, which raised this verifier's
workflow-completion wait from 90 s to 300 s ("for cold GH runners +
npm ci+ glob").The change reached
mainin the #264 release, and the diff on this file is exactly- timeout = 90/+ timeout = 300. At 90 s the loop always finished wellinside the subprocess budget; at 300 s it exactly meets it, so the headroom is gone.
Related but not a fix: open PR #223 adds an explicit
except subprocess.TimeoutExpiredaround this call, reporting "Verification script timed out after 300s". That makes the
failure legible, but it keeps
timeout=300, so the run still ends without a verdict.It only bites when the awaited CI state does not settle quickly: in our runs 22 of
25 verified in 5–43 s, and 3 stalled and were killed at ~305 s, losing their grade.
Neither number is configurable — the 300 in
run_verificationis a literal,repeated across six task managers, and github uses the base class's.
Two ways to close it, either is fine:
run_verification's timeout above the longest verifier poll, ideallyderiving one from the other rather than repeating a constant.
Happy to send a PR for whichever you prefer.