[workflow-analysis] Weekly Workflow Analysis — 2026-10-05: Issue Monster failing 100%, plus a cross-workflow burst incident 04:52–06:07 UTC

[github-actions[bot]]

Overview

Sampled the most recent ~6 hours of GitHub Actions run history for this repo (2026-10-05 03:46–09:46 UTC, 400 runs) via the GitHub MCP server, after the agenticworkflows logs bulk sweep across all 320 workflows repeatedly stalled (20+ min with no progress — see Methodology note below). The headline finding is a sharp, time-boxed reliability incident, plus one workflow with a 100%-failure root cause unrelated to that incident.

Key metrics

Conclusion	Count	Share
success	199	49.8%
action_required	98	24.5%
failure	59	14.8%
skipped	37	9.2%
cancelled	4	1.0%
startup_failure	1	0.25%

• Burst incident, 04:52–06:07 UTC: run volume and failure rate both spiked (up to 100 runs in a single 13-minute window vs. ~25-30/hour baseline), with the non-success rate hitting 60-85% during the spike vs. ~25-32% before/after. Many unrelated workflows failed at the exact same second (e.g. "Agentic Commands" and "AI Moderator" both failed at 04:51:28 on the same PR branch), which points to a shared infrastructure blip (runner contention / MCP gateway hiccup) rather than independent per-workflow bugs.
• "Issue Monster" failed all 12/12 scheduled runs today (hourly, 01:03–09:35 UTC), each with http_400_response_error, exitCode=1, "Agent reported that it could not complete the task" (infrastructure_error). This is a distinct, ongoing root cause — not part of the burst — and affects every single run, all day. It shares its inference engine (pi) with "AI Moderator", which also appears repeatedly in the failure list.
• "action_required" is often not a failure: most of these cluster on copilot/fix-* PR branches for workflows like "Agentic Commands" and "AI Moderator" — consistent with GitHub's approval gate for externally-authored/first-run workflow executions, not a code defect. Worth excluding from failure-rate alerting to avoid overcounting.
• Top problem workflows (failure + action_required + startup_failure, out of total runs in the sampled window):

Workflow	Problem / Total
Agentic Commands	34 / 51
CGO	12 / 13
AI Moderator	11 / 19
Squad	11 / 11
Content Moderation	9 / 11
CWI	9 / 12
Issue Monster	8 / 8 (today's full sample: 12/12)
Work Queue Compilation Integration	6 / 7
CJS	6 / 8

Next actions

1. Fix "Issue Monster" (engine: pi) — investigate the HTTP 400 from its inference call (likely a bad/deprecated model id, malformed request, or auth issue); it has failed every run today and is burning compute for zero output.
2. Check "AI Moderator" for the same root cause — it shares the pi engine and shows a similar failure signature.
3. Correlate the 04:52–06:07 UTC burst against GitHub Actions platform status / runner capacity rather than debugging each affected workflow individually — the simultaneous, cross-workflow failure timestamps suggest one shared cause.
4. Exclude action_required from failure-rate dashboards, or split it out separately from failure/startup_failure, since a large share reflects expected approval-gating on Copilot-authored PR branches.
5. Once "Issue Monster" is fixed, apply the audit tool's optimization suggestion: it's a narrow, read-only "issue_response" task that likely doesn't need a frontier model — try gpt-4.1-mini or claude-haiku-4-5 to cut cost.

View Details — methodology and raw evidence

• This repo runs 320 active agentic workflows (315 active / 5 disabled per gh aw status), spanning engines: copilot (130), codex (77), claude (58), pi (32), aider (4), goose (4), opencode (3), crush (2), cursor (2), deepseek-harness (2), gemini (2), kiro (2), pydantic-ai (2).
• A full agenticworkflows logs --start_date -1w sweep across all workflows was attempted twice (with and without an outer timeout) and did not complete after 20+ minutes each time — likely too expensive given the workflow count and per-run log/artifact volume. Pivoted to the GitHub Actions REST API via the read-only GitHub MCP server instead.
• A status=failure-filtered run list showed a cluster of 100 failures in the most recent ~9 hours, then jumped straight to a cluster from 2026-02-18/19/20 on the next page — a 7+ month gap. This is not a confirmed "zero failures for 7 months" claim; it may reflect API pagination/caching behavior (total_count appeared capped at 2500) as much as true history, so it's reported here only as a caveat, not a finding.
• Root-cause evidence for Issue Monster, from job logs (run §37291067866):

  ##[error]Agent reported that it could not complete the task
  [51] untimed execution/agent agent.execution categories=["http_400_response_error"] errorCodes=[400] errorTypes=[] exitCode=1
  

• Correlated burst-window failures, same second across different workflows on the same PR branch (copilot/fix-linter-flags-omission): §37265294850 (Agentic Commands, failure) and run 37265294729 (AI Moderator, failure), both created_at: 2026-10-05T04:51:28Z.
• Did not obtain token/cost usage for this period — the tool path that normally surfaces that (agenticworkflows logs) is the one that stalled; worth a follow-up with a narrower per-workflow query (e.g. workflow_name + small count) rather than a repo-wide sweep.

References:

• §37291067866 — Issue Monster, latest failure (http_400_response_error)
• §37265294850 — Agentic Commands, burst-window failure
• §37265339163 — Agentic Commands, action_required (approval-gated, not a code failure)

Generated by 🔍 Weekly Workflow Analysis · claude · agent · 339.9 AIC · ⌖ 7.49 AIC · ⊞ 6.3K · ◷

• expires on Oct 6, 2026, 1:53 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-06T09:53:43.686Z.

Closed by Workflow