docs(rfc): queue-scoped speculation design#413
Conversation
e0ac903 to
55dd290
Compare
sbalabanov
left a comment
sbalabanov
left a comment
There was a problem hiding this comment.
back to you based on brainstorming session
55dd290 to
b98e465
Compare
|
|
||
| ### The default Speculator | ||
|
|
||
| Composed in wiring from two swappable interfaces — swap the whole Speculator for a joint optimizer or learned model, or tune behavior by replacing just the Generator or Allocator. Its entire body: open the candidate stream over the batches and their path sets, then hand the stream and the path sets to the Allocator. |
There was a problem hiding this comment.
swap the whole Speculator for a joint optimizer or learned model, or tune behavior by replacing just the Generator or Allocator
I feel like this is unnecessary, the extension model implies this, which begs the question: why are generator and allocator not extensions by themselves? Don't we intend to try and swap those out?
There was a problem hiding this comment.
They are swappable, but intentionally remain composition points inside the default Speculator rather than controller-facing extensions. The controller depends only on the Speculator contract, allowing an alternative joint optimizer to replace it without preserving the Generator/Allocator split. I clarified this in the RFC.
| Composed in wiring from two swappable interfaces — swap the whole Speculator for a joint optimizer or learned model, or tune behavior by replacing just the Generator or Allocator. Its entire body: open the candidate stream over the batches and their path sets, then hand the stream and the path sets to the Allocator. | ||
|
|
||
| - **Generator** — the queue-wide ordered source of candidate paths. Given the batches and their path sets, it returns a single iterator that yields coherent, depth-capped candidates best-first across every head. Scoring and depth capping are internal. It prices each candidate as the product of its dependencies' landing chances: a dependency's chance starts at its `Batch.Score` (forced to 1 or 0 once resolved). Build evidence from its path set then sharpens it: a dependency whose speculative build has passed is likelier to land. It skips candidate paths already terminal in the path sets (passed, failed, cancelled), so best-first never re-emits a dead future. It caps **depth** — the number of unresolved dependencies a head's paths range over — so one deep chain can't blow up a run. A head beyond the cap is skipped this run and enters as its dependencies resolve. | ||
| - **Allocator** — spends the build budget (the queue's cap on concurrent builds) over the single iterator. It pulls candidates in order until the budget fills. It matches each against the path sets by ID: a path already building is kept, not re-triggered. Building and cancelling paths charge the budget (a cancelling build holds CI until observed terminal); passed, failed, and cancelled paths charge none. The sticky default fills only free slots and leaves every viable in-flight build running; the preempting default cancels in-flight paths that fall below the funded set. Budget is the only rationing lever: there is no score floor, so the default spends every slot on the best remaining candidates, however long the shot. |
There was a problem hiding this comment.
Building and cancelling paths charge the budget (a cancelling build holds CI until observed terminal)
This is interesting.. let's say I have a 15 build budget, 14 ongoing. I speculate, and note that I can cancel 10 builds. Given that there is only 1 build left in the budget, that means I can't return more than 1 build to start, even though I know the budget will be there shortly (given I'm canceling 10 of em). Seems suboptimal that we'd have to re-run the speculator to make use of that budget?
There was a problem hiding this comment.
Yes, this intentionally requires another run. Cancellation is best-effort and the CI slot remains occupied until terminal; allocating against expected capacity could exceed the hard cap if cancellation is slow or loses a race. The terminal result publishes dirty, so the next run fills the released slots. I clarified this in the RFC.
|
|
||
| ### Speculator | ||
|
|
||
| The controller calls the one extension, `Speculator.Speculate(batches, path sets)`, which returns paths to build and in-flight paths to preempt — nothing else. The controller validates that result, dropping any refuted, incoherent, or already-failed build. |
There was a problem hiding this comment.
Can we document an rough time-deadline for the Speculate call? Our controller should implement a bailout if a speculate call is taking too long (and effectively misbehaving).
There was a problem hiding this comment.
I would prefer not to enforce a Speculator-specific deadline in the controller. Like our other extension calls, the controller passes its context and the implementation is responsible for honoring cancellation and keeping the call within its operational budget. That leaves each implementation free to make the right latency/quality tradeoff instead of baking an arbitrary timeout into the contract; we can instrument duration and optimize or bound a concrete implementation if it becomes necessary.
## Summary ### Why? A merge queue that verifies one change at a time is bounded by its slowest build. Speculation builds a batch early against a guess at how its unresolved dependencies will resolve, so a matching build is ready when reality settles while merging stays strict. Getting that right on an eventually consistent, transactionless store needs an explicit design for how a path is represented, which paths build under a bounded CI budget, what is persisted, and where each decision lives. ### What? This RFC designs speculation for SubmitQueue end to end. Each queue change fires one run per queue: the speculate controller reads the current facts once, reconciles path state, refutes broken bets, finalizes verdicts, asks one pluggable extension which paths to build or preempt, and writes the result. Only chosen paths are persisted; every other possibility is recomputed each run. The Speculator is the only controller-facing extension. It chooses which paths to build and which running ones to cancel, while verdicts and correctness cancels remain controller rules. The default Speculator internally composes a Generator, which produces a lazy best-first candidate stream, and an Allocator, which spends the CI budget and settles in-flight paths by ID; alternate Speculators need not preserve that split. A path is self-describing: it carries an included, excluded, or dropped bet on every dependency. Its identity hashes the head plus its bets, build dispatches are replayed idempotently by path and attempt, and pending, building, and cancelling paths continue to charge the hard CI budget until terminal. The design also covers conflict relaxation and bypassing a large dependency once passed builds cover every possible resolution. The RFC fills the previously empty `doc/rfc/submitqueue/speculation.md`.
b98e465 to
a3f6bf6
Compare
3 participants
Summary
Why?
A merge queue that verifies one change at a time is bounded by its slowest build. Speculation builds a batch early against a guess at how its unresolved dependencies will resolve, so a matching build is ready when reality settles while merging stays strict. Getting that right on an eventually consistent, transactionless store needs an explicit design for how a path is represented, which paths build under a bounded CI budget, what is persisted, and where each decision lives.
What?
This RFC designs speculation for SubmitQueue end to end. Each queue change fires one run per queue: the speculate controller reads the current facts once, reconciles path state, refutes broken bets, finalizes verdicts, asks one pluggable extension which paths to build or preempt, and writes the result. Only chosen paths are persisted; every other possibility is recomputed each run.
The Speculator is the only controller-facing extension. It chooses which paths to build and which running ones to cancel, while verdicts and correctness cancels remain controller rules. The default Speculator internally composes a Generator, which produces a lazy best-first candidate stream, and an Allocator, which spends the CI budget and settles in-flight paths by ID; alternate Speculators need not preserve that split.
A path is self-describing: it carries an included, excluded, or dropped bet on every dependency. Its identity hashes the head plus its bets, build dispatches are replayed idempotently by path and attempt, and pending, building, and cancelling paths continue to charge the hard CI budget until terminal. The design also covers conflict relaxation and bypassing a large dependency once passed builds cover every possible resolution.