Conversation
The lowering and the compiler already turned a `for` into a loop over `GetIter`, `Next` and `Exhausted`. The interpreter stubbed all three, and this fills them in. The end of a walk is a value rather than an exception. CPython raises `StopIteration`, which makes every `for` loop an exception handler; here `Next` writes a sentinel and `Exhausted` tests it, so a loop is a test and a branch. No program can obtain one, because nothing else constructs it and the only instruction that reads it is emitted immediately after the one that writes it. An iterator holds the container and a position, which is what CPython does and is observable: a list that grows while walked keeps the walk going, one that shrinks ends it early, and a dict or set that changes size raises `RuntimeError`. The dict position is into the entry table rather than a count of live entries, so a deletion earlier in the table cannot shift what comes next. Strings are the exception and are widened once, because the nth code point of UTF-8 is only reachable by counting from the front. `range`, `len`, `iter` and `next` come with it. A range is never built and holds integers of any size. `len` now covers all six containers plus `range`, which is what it was waiting for. A builtin knows whether it is a function or a type, so `print(range)` says `<class 'range'>`. The first version of the range walk computed the nth value from n, which is a multiply and a division per step and cost about twice what the same loop written with `while` cost. It adds the step to a running value now.
Both of these only concern the members of the right hand side, and the general in place path was going through the ordinary operator and then cloning the result back over the target, which is two full copies of the left set per step. Growing a set one member at a time was quadratic: 5000 members took 0.77s, 10000 took 3.0s, 20000 took 10.7s, and 100000 did not finish in a time worth waiting for. Inserting or removing only the right hand side's members brings the 100000 case to 0.04s. `&=` and `^=` keep the general path. Those have to look at every member of the left whatever they do, so there is nothing to win there and CPython pays the same. The right hand side is read out before the left is borrowed, because `s |= s` is a program somebody writes and both sides can be the same set. A right hand side that is not a set at all still falls through to the ordinary operator, which is what raises the `TypeError`. Found by benchmarks/tier0/iterate.py in kohebi-bench, which hung on the set section while CPython finished the whole file in half a second.
tamnd
added a commit
to tamnd/kohebi-bench
that referenced
this pull request
Aug 29, 2026
Fresh reports from both, four runtimes on the i9 and three on the Air, now including the new `iterate` benchmark and with the in place set fix from tamnd/kohebi#94 in the binary. They disagree by a factor of two. kohebi is 1.37x CPython on the Air and 0.79x on the i9, on the same fourteen benchmarks. Going from the Air to the i9 makes CPython 2.1x to 3.2x faster on every benchmark and makes kohebi 1.19x to 1.28x faster on the four that are pure loop, which is about the clock ratio between the two chips and nothing more. Ruled out: codegen. Rebuilding on the i9 with `-C target-cpu=native` instead of the generic x86-64 baseline moves none of the three loop benchmarks beyond run to run variation. results/README.md now says all of that, says which of the two numbers to work against until it is settled, and says that the answer is in a profile rather than in another table.
tamnd
added a commit
to tamnd/kohebi-bench
that referenced
this pull request
Aug 29, 2026
…DTrace probes (#5) * Add a tier zero benchmark that walks containers with `for` Every other program in benchmarks/tier0 counts with `while`, because that was all the runtime could execute when they were written. A `for` loop is not the same work: it asks a container for an iterator, steps it, and tests whether the step found anything, and what that costs depends entirely on whether the iterator was built once or is being rebuilt per step. Seven sections. A range counting up and a range counting down by three, because the direction and the stride are separate arms in anything that resolves one, and because a runtime that computes the nth value from n pays a multiply and a division on every step of the loop most Python programs are made of. Then a half million element list, an eight element tuple walked two hundred thousand times, an 8600 character string, a hundred thousand entry dict and a hundred thousand member set. The string is there because the nth code point of UTF-8 is only reachable by counting from the front, so a runtime that steps it that way is quadratic in the length, which looks fine on the short strings in a test. The dict and the set are there because their iterators cannot safely hold a pointer into the table across a step and have to hold a position instead, and a position into a table with holes in it is where an off by one skips an element rather than crashing. It has already earned itself. It hung under kohebi, which turned out to be `s |= {x}` rebuilding and re-copying the whole set on every step, quadratic in the size of the set. Fixed in tamnd/kohebi, and kohebi now runs the whole file in 0.46s against CPython 3.14's 0.54s. * Record the two machines, which do not agree about kohebi Fresh reports from both, four runtimes on the i9 and three on the Air, now including the new `iterate` benchmark and with the in place set fix from tamnd/kohebi#94 in the binary. They disagree by a factor of two. kohebi is 1.37x CPython on the Air and 0.79x on the i9, on the same fourteen benchmarks. Going from the Air to the i9 makes CPython 2.1x to 3.2x faster on every benchmark and makes kohebi 1.19x to 1.28x faster on the four that are pure loop, which is about the clock ratio between the two chips and nothing more. Ruled out: codegen. Rebuilding on the i9 with `-C target-cpu=native` instead of the generic x86-64 baseline moves none of the three loop benchmarks beyond run to run variation. results/README.md now says all of that, says which of the two numbers to work against until it is settled, and says that the answer is in a profile rather than in another table. * Refuse a CPython baseline built with the DTrace probes The two machines disagreed by a factor of two about kohebi, 1.37x CPython on the MacBook Air and 0.79x on the i9, on the same fourteen benchmarks. That is not a gap between two chips and it was not kohebi. Homebrew builds CPython with --with-dtrace. macOS always has DTrace, so the probes are real code in the eval loop rather than the nothing they compile to on a Linux box without the systemtap headers. Same Mac, same 3.14.7, five million times around a `while` loop: 0.80s from Homebrew's build and 0.24s from one without the probes. Apple's own 3.9.6 does it in 0.45s, and a five year old Python beating the current one by nearly two is the tell. Every number published from that machine made kohebi look about three times better than it is. `Runtime.crippled` asks CPython how it was built and refuses before anything is timed, which is what `Runtime.misidentified` already does for which interpreter it is. A debug build is refused everywhere; the DTrace build only on macOS, because the Debian build on the other machine has the same flag and measures the same as one without it. Refused rather than warned about. A warning on stderr scrolls past and the report it produced does not say anything is wrong with it. results/mba-m4 is re-run against an honest baseline and now agrees with the i9 about the shape of things: kohebi is between a half and level with CPython on work, ahead on startup, at about a third of its memory. results/README.md has all of it, including what is left of the gap between the two machines and what has been ruled out.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The lowering and the compiler already turned a
forinto a loop overGetIter,NextandExhausted. The interpreter stubbed all three, and this fills them in.The end of a walk is a value rather than an exception. CPython raises
StopIterationthere, which makes everyforloop in the language an exception handler and makes every iteration pay a little towards the one that will not happen. HereNextwrites a sentinel andExhaustedtests it, so a loop compiles to an ordinary test and branch. No Python program can obtain one, because nothing else constructs it and the only instruction that reads it is emitted immediately after the one that writes it. When generators arrive they raiseStopIterationthe way Python says and the runtime turns that back into this at the boundary.An iterator holds the container and a position rather than a copy, which is what CPython does and is observable in both directions. A list that grows while it is being walked keeps the walk going, one that shrinks ends it early, and a dict or a set that changes size raises
RuntimeErrorinstead. The position into a dict is an index into the entry table rather than a count of live entries, so a deletion earlier in the table cannot silently shift what comes next. Strings are the one container walked from a copy, because the nth code point of UTF-8 is only reachable by counting from the front and stepping one that way would make walking a string quadratic.range,len,iterandnextcame with it, since they all needed exactly this. A range is never built, sofor i in range(1000000)is one object rather than a million, and it holds integers of any size rather than machine words.lennow covers all six builtin containers plusrange, which is what it was held back for, includinglen(range(2 ** 70))raisingOverflowErrorbecause a length has to fit in a machine word even when the range does not. A builtin now knows whether it is a function or a type, soprint(range)says<class 'range'>.Performance
The first version of the range walk computed the nth value from n, which is a multiply and a division on every step.
for i in range(3000000): total += icost 0.77s against CPython's 0.40s, and the same loop written withwhilecost 0.76s, so the counted loop was paying twice for its counting. Walking with a running value and one comparison brings it to roughly level with CPython. A proper interleaved measurement follows on the quiet machine.Testing
Fifty new tests. Thirty three run whole programs and assert what a running CPython 3.14 prints for them, including every error message and the two places CPython does not agree with itself about them. Eight are unit tests over the range arithmetic, which is where the rounding is easy to get wrong in a way no program test would notice until the last element of some loop went missing.
Full gate green:
cargo fmt,cargo test --workspace,cargo clippy --workspace --all-targets --all-features,cargo docwith warnings denied.Part of #7.