Skip to content

fix(automation): stop mention sweep after shared rate-limit exhaustion - #1012

Open
seonghobae wants to merge 4 commits into
mainfrom
fix/agent-mention-rate-limit-fail-fast-20260815
Open

fix(automation): stop mention sweep after shared rate-limit exhaustion#1012
seonghobae wants to merge 4 commits into
mainfrom
fix/agent-mention-rate-limit-fail-fast-20260815

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Incident

Scheduled Review Agent Mention Router run 31868885733 exhausted the OpenCode GitHub App installation REST budget and then continued traversing organization repositories. The run produced zero dispatches and 116 isolated failures, including .github#1008 and bandscope, after the first installation-wide API rate limit exceeded response.

Test-first repair

  • RED commit e5e255272f0cef3ed9c4b0b365d5df793cf46807 adds focused regressions proving primary installation-budget exhaustion must stop before a second PR is touched and that secondary-rate-limit messages are also sweep-global.
  • GREEN commit 602d4f91f5b58657f1d8e00c18ae0b2a20900280 adds SweepRateLimitExhausted plus exact message classification and raises after recording the first exhausted scope.
  • Doctoring commit 2502f665a1b74cb7814c01535c3dbaf3a20ce259 records the incident, trust boundary, rollback, and APA 7 references to GitHub's primary/secondary REST and GitHub App rate-limit guidance.

Boundary

The repair does not retry, sleep, widen permissions, rotate credentials, alter reviewer identity, change the canonical invocation key, weaken the exact-name artifact ledger, or represent failed work as dispatched. Ordinary candidate-local failures remain isolated. Explicit GitHub primary/secondary exhaustion stops only the current scheduled sweep so it cannot amplify an already exhausted shared installation budget.

This is complementary to concurrency-isolation PR #835 and dispatch-envelope PR #840; it does not duplicate their queue or payload changes.

Exact identity

  • Protected base: main@6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba
  • Exact head at PR creation: 2502f665a1b74cb7814c01535c3dbaf3a20ce259
  • Changed paths: scripts/ci/agent_mention_sweep.py, tests/test_agent_mention_rate_limit.py, docs/doctoring/agent-mention-rate-limit-fail-fast.md

Merge gate

Keep Draft until exact-current-head Agent Mention Router Quality CI, 100% owned statement/branch/docstring coverage, security/SAST/supply-chain checks, current semantic review, zero valid unresolved threads, two qualifying non-author formal approvals including last-push approval, and protected-main branch rules are all satisfied. No queued, failed, predecessor-head, self/author, or bypass evidence is acceptance.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: dfff45a0-60e5-4a95-af78-b5f905758a20

📥 Commits

Reviewing files that changed from the base of the PR and between bbedc1a and 0cccd7f.

📒 Files selected for processing (3)
  • docs/doctoring/agent-mention-rate-limit-fail-fast.md
  • scripts/ci/agent_mention_sweep.py
  • tests/test_agent_mention_rate_limit.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review August 15, 2026 11:35
@seonghobae
seonghobae enabled auto-merge (squash) August 15, 2026 11:35

Copy link
Copy Markdown
Contributor Author

Exact-current-head read-only review request for 2502f665a1b74cb7814c01535c3dbaf3a20ce259 over protected base 6eb06cdd08c79a06f7b390069d4ffa49e2eb7dba.

This focused TDD repair stops an organization-wide mention sweep immediately after the first explicit GitHub primary or secondary installation-rate-limit exhaustion, while preserving isolation for ordinary candidate-local failures. Exact-head Agent Mention Router Quality CI and the hosted security, SAST, supply-chain, and dependency checks are terminal-success; unresolved inline threads are zero. The PR is now Ready and auto-merge is armed subject to independent approvals and protected-main rules.

Review this exact head only. Preserve reviewer identities and credentials; do not mutate the branch, synthesize author approval, weaken protection, or merge.

@opencode-agent review
@cwl-noema-review review
@strix review

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review @coderabbitai review

Review exact head 2502f665a1b74cb7814c01535c3dbaf3a20ce259. Verify that only explicit GitHub primary/secondary rate-limit exhaustion becomes sweep-global, the first exhausted scope is recorded exactly once, traversal stops before any later PR/repository request, ordinary candidate-local failures remain isolated, no retry/sleep/credential rotation/permission widening is introduced, and the change preserves canonical dispatch, ledger, reviewer, and merge-authority contracts. Submit a formal exact-head verdict.

@coderabbitai

coderabbitai Bot commented Aug 15, 2026

Copy link
Copy Markdown

Rate Limit Exceeded

@seonghobae have exceeded the limit for the number of chat messages per hour. Please wait 39 minutes and 3 seconds before sending another message.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The fail-fast direction is right, but this head is not yet safe to merge.

is_rate_limit_exhaustion looks for the contiguous phrase api rate limit exceeded. GitHub also returns API rate limit already exceeded. That string does not contain the needle, so the sweep would keep walking repositories and recreate the 116-failure amplification.

The tests also mock away list_recent_pull_requests, which is the incident path. main() still lets SweepRateLimitExhausted escape as a traceback, so the scheduled job does not tell the operator to wait for the installation budget reset.

Do not merge this head. Land the classification, incident-path, and CLI-exit repair on cursor/bc-5bb76fb1-67aa-4840-8b3a-1fe7792dd979-6966 (or cherry-pick that commit onto this branch), then re-run Agent Mention Router Quality CI on the new exact head.

Open in Web View Automation 

Sent by Cursor Automation: fix all


message = " ".join(str(error).split()).casefold()
return (
"api rate limit exceeded" in message

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This contiguous needle misses API rate limit already exceeded. That GitHub wording inserts already between limit and exceeded, so the first exhausted repository would be recorded as an isolated skip and later repositories would keep consuming the empty installation budget.

Match rate limit plus exceeded/exhausted, or secondary rate limit. Add a regression that feeds the already exceeded string and proves the next repository is never requested.

counters.failures += 1
message = " ".join(str(error).split()) or error.__class__.__name__
print(
f"::warning::Agent mention sweep skipped {scope}: {message[:1000]}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still says the scope was skipped, then raises. Operators will treat it as a local skip and re-run. Emit ::error:: with the next action: wait for the installation REST budget to reset; do not re-run this sweep immediately. Catch SweepRateLimitExhausted in main() and return 1 so the scheduled job fails closed without a traceback.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-current-head review request for 767432db066bc84c3dcc4a94a2b55cf3f83653d8. The fix now classifies both GitHub primary-limit message forms, stops the sweep globally, and catches the CLI boundary to emit the budget-reset action with exit 1. Targeted regressions: 9 passed; full suite on the rebased source: 1119 passed, 16 subtests passed. Please review this exact head only; no self-approval or bypass merge.

@seonghobae
seonghobae force-pushed the fix/agent-mention-rate-limit-fail-fast-20260815 branch from 767432d to 42715e9 Compare August 19, 2026 09:44
@seonghobae

Copy link
Copy Markdown
Contributor Author

Rebased onto current main 9e9f59f3. Rate-limit regression tests passed (3 passed); git diff --check passed. Project #1 is In Progress.

@seonghobae
seonghobae force-pushed the fix/agent-mention-rate-limit-fail-fast-20260815 branch from 42715e9 to 0cccd7f Compare August 19, 2026 15:00
@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head evidence: rebased the agent-mention rate-limit fail-fast repair onto main bbedc1a. Exact head 0cccd7f. Focused rate-limit tests: 3 passed; git diff --check passed. Protected hosted checks remain; no bypass used.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

1 participant