fix(automation): stop mention sweep on already-exceeded rate limits

ARCHITECTURE.md

@@ -74,6 +74,29 @@ sequenceDiagram
   MS->>PR: merge only on current-head approval + green checks
 ```
 
+## Mention-sweep shared-budget stop
+
+```mermaid
+flowchart TD
+  Router["Review Agent Mention Router"]
+  List["List org repos and recent PRs"]
+  Work["Build mentions and dispatch"]
+  Cap{"Shared installation REST budget exhausted?"}
+  Halt["Record one scope, tell the operator to wait, exit 1"]
+  Next["Continue later repos and PRs"]
+
+  Router --> List --> Work --> Cap
+  Cap -->|"yes"| Halt
+  Cap -->|"no"| Next
+  Next --> Work
+```
+
+The scheduled sweep treats GitHub primary and secondary rate-limit wording as
+one shared installation budget, not a per-repository skip. After the first
+exhausted scope it stops so later repositories cannot amplify an already empty
+budget. Ordinary candidate-local failures stay isolated. Operators wait for
+GitHub to reset the installation window; they do not re-run immediately.
+
 ## Trust boundaries
 
 - Required review workflows execute **base-branch** scripts. A PR that edits
@@ -106,5 +129,7 @@ tests pin workflow structure and governance prose so drift fails closed.
   contract.
 - [`docs/doctoring/hourly-nvidia-nim-autofix.md`](docs/doctoring/hourly-nvidia-nim-autofix.md)
   — current increment's repair-worker decision and APA 7th citations.
+- [`docs/doctoring/agent-mention-rate-limit-fail-fast.md`](docs/doctoring/agent-mention-rate-limit-fail-fast.md)
+  — mention-sweep shared-budget stop, incident-path tests, and APA 7th citations.
 - [`docs/doctoring/fast-mlsirm-hourly-review-caller.md`](docs/doctoring/fast-mlsirm-hourly-review-caller.md)
   — product-specific psychometric repair heartbeat and scientific gates.

CHANGELOG.md

@@ -26,6 +26,7 @@ Semantic Versioning where the repository publishes a release.
 
 ### Fixed
 
+- Stop the organization mention sweep after the first shared GitHub primary or secondary rate-limit exhaustion, including the `API rate limit already exceeded` wording, the repository-listing incident path, and dispatch-time exhaustion. The scheduled CLI now exits `1` with an operator next action instead of continuing the 116-failure amplification or surfacing only a traceback.
 - Materialized base Python locks only when every package line is an exact SHA-256 pin or a bounded relative `-r`/`--requirement` include. A lone `--require-hashes` directive, a dotted include such as `./lock.txt`, or `-r other-hashes.txt` no longer enters the trusted build context.
 - Refused a conflict-scope repository root whose immediate parent is a symbolic link, so a swapped parent cannot redirect the canonical worktree after the last-component check (CWE-367).
 - Bounded the Strix quality self-test's deterministic timeout fixtures to 3-second process and 5-second fake-sleep budgets so exact-head policy evidence completes inside the existing job limit without changing production Strix scanner timeouts, providers, credentials, or review semantics.

docs/doctoring/agent-mention-rate-limit-fail-fast.md

@@ -0,0 +1,66 @@
+# Agent mention sweep rate-limit fail-fast boundary
+
+Updated: 2026-08-16
+
+## Incident
+
+Scheduled `Review Agent Mention Router` run `31868885733` exhausted the OpenCode GitHub App installation REST budget before processing the requested review queue. The sweep continued traversing repositories after the first installation-wide `API rate limit exceeded` response and finished with zero dispatches plus 116 isolated failures. Repeating requests after the shared budget is exhausted cannot recover candidate-local work and consumes runner time while obscuring the single control-plane cause.
+
+## Decision
+
+Treat explicit GitHub primary- or secondary-rate-limit messages as **sweep-global capacity exhaustion**, not candidate-local failures. The sweep records the first failed scope, emits an operator `::error::` with the next action (wait for the installation REST budget to reset; do not re-run immediately), then raises `SweepRateLimitExhausted`. The scheduled CLI catches that exception and exits `1` so the workflow fails closed without a raw traceback. Ordinary repository, pull-request, review, acknowledgement, and dispatch failures remain isolated exactly as before.
+
+Classification matches GitHub's documented wording families, including `API rate limit exceeded`, `API rate limit already exceeded`, and secondary-limit messages. A contiguous `"api rate limit exceeded"` needle is not sufficient: the GraphQL/already-exhausted phrasing inserts `already` between `limit` and `exceeded`, and that wording must also stop the sweep.
+
+This change is intentionally narrow. It does not retry, sleep, change credentials, widen permissions, alter the canonical invocation key, modify the exact-name artifact ledger, or claim that a failed request was dispatched. A later scheduled invocation may run after GitHub restores capacity. Interactive/local routing and the separate concurrency-isolation repair remain independent control-plane lanes.
+
+```mermaid
+flowchart TD
+  Sweep["Organization mention sweep"]
+  Err{"Exception on repo, PR, or dispatch?"}
+  Local["Record isolated failure and continue"]
+  Shared{"Primary or secondary rate-limit wording?"}
+  Stop["Record one scope, emit next-action error, raise SweepRateLimitExhausted"]
+  Exit["CLI exits 1; do not re-run until budget reset"]
+
+  Sweep --> Err
+  Err -->|"no"| Sweep
+  Err -->|"yes"| Shared
+  Shared -->|"no"| Local
+  Local --> Sweep
+  Shared -->|"yes"| Stop
+  Stop --> Exit
+```
+
+## Why fail-fast
+
+GitHub documents that installation access tokens share an installation-level primary REST budget. When a primary limit is exceeded, requests return HTTP 403 or 429 and callers should not retry until the reset time. GitHub also states that integrations should stop and wait on secondary-rate-limit responses; continuing to make requests while rate-limited may lead to integration bans. The current sweep cannot safely infer reset headers from the `gh` exception string, so the bounded action is to stop the current scheduled traversal rather than amplify the exhausted state.
+
+## Verification contract
+
+- a synthetic installation-wide primary-limit error on the first PR aborts before the second PR is touched;
+- the incident path (first repository pull listing exhausted) records exactly one failure and never requests a later repository;
+- a dispatch-time primary-limit error aborts before the next pull request is built;
+- exactly one failure is recorded for the first exhausted scope;
+- secondary-limit, `already exceeded`, and HTTP 429 secondary messages are classified as sweep-global exhaustion;
+- unrelated authorization/resource errors remain candidate-local and preserve existing failure isolation;
+- `main()` returns `1` with an `::error::` next action when `SweepRateLimitExhausted` is raised;
+- the permanent agent-mention quality suite continues to require 100% owned production statement/branch and public docstring coverage.
+
+## Operator next action
+
+If the scheduled sweep fails with `::error::` and `rate limit`, wait for GitHub to restore the installation REST budget. Do not re-run the workflow immediately. The next hourly schedule is the recovery path. Ordinary isolated `::warning::` skips are not this signal.
+
+## Rollback
+
+Revert `SweepRateLimitExhausted`, `is_rate_limit_exhaustion`, the CLI catch, and their focused regressions if GitHub changes the CLI error contract or the router gains structured response-header handling. Do not restore repeated API calls after a proven shared rate-limit exhaustion without an equivalent bounded backoff/stop mechanism.
+
+## References
+
+GitHub. (n.d.-a). *Rate limits for the REST API*. GitHub Docs. Retrieved August 16, 2026, from https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api
+
+GitHub. (n.d.-b). *Rate limits for GitHub Apps*. GitHub Docs. Retrieved August 16, 2026, from https://docs.github.com/en/apps/creating-github-apps/registering-a-github-app/rate-limits-for-github-apps
+
+GitHub. (n.d.-c). *Best practices for using the REST API*. GitHub Docs. Retrieved August 16, 2026, from https://docs.github.com/en/rest/using-the-rest-api/best-practices-for-using-the-rest-api
+
+GitHub. (n.d.-d). *Rate limits and query limits for the GraphQL API*. GitHub Docs. Retrieved August 16, 2026, from https://docs.github.com/en/graphql/overview/rate-limits-and-query-limits-for-the-graphql-api

scripts/ci/agent_mention_sweep.py

@@ -30,6 +30,21 @@ class SweepMetrics:
     failures: int = 0
 
 
+class SweepRateLimitExhausted(RuntimeError):
+    """Signal that shared GitHub API capacity is unavailable for this sweep."""
+
+
+def is_rate_limit_exhaustion(error: Exception) -> bool:
+    """Return whether an API error says the shared primary/secondary budget is exhausted."""
+
+    message = " ".join(str(error).split()).casefold()
+    if "secondary rate limit" in message:
+        return True
+    return "rate limit" in message and (
+        "exceeded" in message or "exhausted" in message
+    )
+
+
 def parse_timestamp(value: str) -> datetime:
     """Parse one GitHub ISO-8601 timestamp into timezone-aware UTC."""
 
@@ -301,10 +316,20 @@ def sweep(
     dispatched = 0
 
     def record_failure(scope: str, error: Exception) -> None:
-        """Record one isolated error and preserve the remaining sweep."""
+        """Record isolated errors but stop when the shared API budget is exhausted."""
 
         counters.failures += 1
         message = " ".join(str(error).split()) or error.__class__.__name__
+        if is_rate_limit_exhaustion(error):
+            print(
+                f"::error::Agent mention sweep stopping at {scope}: shared GitHub "
+                "API rate limit exhausted. Wait for the installation budget to "
+                "reset; do not re-run this sweep immediately."
+            )
+            raise SweepRateLimitExhausted(
+                "GitHub API rate limit exhausted; stopping organization sweep "
+                "to preserve the shared installation budget"
+            ) from error
         print(
             f"::warning::Agent mention sweep skipped {scope}: {message[:1000]}"
         )
@@ -374,19 +399,26 @@ def main(argv: Sequence[str] | None = None) -> int:
         os.environ.get("OPENCODE_REPOSITORY_DISPATCH_TARGETS", "")
     )
     metrics = SweepMetrics()
-    sweep(
-        target_client=GitHubClient(
-            os.environ.get("TARGET_REPOSITORY_TOKEN", "")
-        ),
-        dispatch_client=GitHubClient(os.environ.get("AGENT_DISPATCH_TOKEN", "")),
-        organization=args.organization,
-        repository_source=args.repository_source,
-        lookback_hours=args.lookback_hours,
-        max_dispatches=args.max_dispatches,
-        opencode_allowlist=allowlist,
-        dry_run=args.dry_run,
-        metrics=metrics,
-    )
+    try:
+        sweep(
+            target_client=GitHubClient(
+                os.environ.get("TARGET_REPOSITORY_TOKEN", "")
+            ),
+            dispatch_client=GitHubClient(os.environ.get("AGENT_DISPATCH_TOKEN", "")),
+            organization=args.organization,
+            repository_source=args.repository_source,
+            lookback_hours=args.lookback_hours,
+            max_dispatches=args.max_dispatches,
+            opencode_allowlist=allowlist,
+            dry_run=args.dry_run,
+            metrics=metrics,
+        )
+    except SweepRateLimitExhausted as exc:
+        print(
+            f"::error::{exc} Wait for the installation REST budget to reset "
+            "before the next scheduled run; do not re-run immediately."
+        )
+        return 1
     return 1 if metrics.failures else 0