Skip to content

Commit 94cb5c0

Browse files
authored
Add a call benchmark (#6)
A call is a frame, an argument list bound to parameters, a body and a return value handed back. Methods, comprehensions, generators and every dunder in the object model are that sequence with something wrapped around it, so whatever a call costs is a floor under all of them. Three shapes, because they cost differently and a regression in one should not hide behind the other two: two positional arguments straight into two registers, a keyword argument that has to be matched against the parameter list, and arguments collected into a tuple for a `*args`. It found something immediately. kohebi was at 0.28s against CPython's 0.08s for 700,000 calls while beating it on the same loop with the call inlined, which turned out to be four heap allocations per call. Three of them are gone in tamnd/kohebi#97 and it is at 0.11s now.
1 parent 07a184e commit 94cb5c0

2 files changed

Lines changed: 58 additions & 3 deletions

File tree

‎README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ Agreement itself is decided in [tamnd/kohebi-compat](https://github.com/tamnd/ko
5858

5959
| Directory | Contents |
6060
| --- | --- |
61-
| `benchmarks/tier0/` | Integer and float loops, branch dispatch, string operators, list growth |
61+
| `benchmarks/tier0/` | Integer and float loops, branch dispatch, string operators, list growth, indexing, iteration, calls |
6262
| `benchmarks/micro/` | Attribute dispatch, integer arithmetic, method calls, homogeneous lists, string building |
6363
| `benchmarks/apps/` | JSON round-tripping, interpreter startup |
6464

@@ -68,9 +68,9 @@ The micro benchmarks are chosen to isolate the specific bets in kohebi's design:
6868

6969
## The tier zero suite, which is the part kohebi can run today
7070

71-
`benchmarks/tier0/` exists because a comparison you cannot run is not a comparison. The tier zero interpreter has assignment, arithmetic, comparison, the boolean operators, `if`, `while`, `break`, `continue`, container displays, `in`, and `print`. It does not have functions, `for` loops, attributes, subscripting, iteration or imports yet, so the five programs in there are written entirely with what exists.
71+
`benchmarks/tier0/` exists because a comparison you cannot run is not a comparison. The tier zero interpreter has assignment, arithmetic, comparison, the boolean operators, `if`, `while`, `break`, `continue`, container displays, `in`, `print`, subscripting, slicing, `for` loops over every builtin container, and functions. It does not have attributes, classes, exceptions or imports yet, so the programs in there are written entirely with what exists.
7272

73-
That makes them a little strange to read. Collatz is inlined rather than called, buckets are five separate names rather than a list, and a list is grown with `items += [i]` because there are no method calls. Every one of them was diffed against CPython 3.14 and PyPy before it went in, so all three runtimes print the same bytes, and none of them is doing less work than the others.
73+
That makes some of them a little strange to read. Buckets are five separate names rather than a list, and a list is grown with `items += [i]` because there are no method calls. The older ones inline what would be a call, which was the only option when they were written and is now a useful contrast against `call.py`. Every one of them was diffed against CPython 3.14 and PyPy before it went in, so all three runtimes print the same bytes, and none of them is doing less work than the others.
7474

7575
These are temporary. When kohebi can run `micro/int_arithmetic.py` as a person would write it, `tier0/int_loop.py` has done its job and comes out. What it buys in the meantime is a real number every week instead of a promise, and the first one already said something useful, which is in `results/`.
7676

‎benchmarks/tier0/call.py‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
"""Calling a function, which is the operation everything else is built on.
2+
3+
A call is a frame, an argument list bound to parameters, a body, and a return
4+
value handed back. Methods, comprehensions, generators and every dunder in the
5+
object model are that same sequence with something wrapped around it, so
6+
whatever a call costs is a floor under all of them.
7+
8+
The three shapes here are the three that cost differently. `add` is the cheap
9+
one: two positional arguments straight into two registers, nothing to search.
10+
`described` is what a keyword argument costs, which is a name matched against
11+
the parameter list before anything can be bound. `sum_of` is a call whose
12+
arguments have to be collected into a tuple first. Splitting them apart means a
13+
regression in one does not hide behind the other two.
14+
15+
The bodies are deliberately trivial. This is not a benchmark about arithmetic
16+
and any real work inside them would only dilute what it is measuring.
17+
18+
Written without imports, because tier zero has none yet.
19+
"""
20+
21+
22+
def add(a, b):
23+
return a + b
24+
25+
26+
def described(value, scale=1, offset=0):
27+
return value * scale + offset
28+
29+
30+
def sum_of(*values):
31+
total = 0
32+
for value in values:
33+
total += value
34+
return total
35+
36+
37+
plain = 0
38+
i = 0
39+
while i < 400_000:
40+
plain = add(plain, 1)
41+
i += 1
42+
43+
keyword = 0
44+
i = 0
45+
while i < 200_000:
46+
keyword = described(keyword, offset=1)
47+
i += 1
48+
49+
collected = 0
50+
i = 0
51+
while i < 100_000:
52+
collected += sum_of(1, 2, 3)
53+
i += 1
54+
55+
print(plain, keyword, collected)

0 commit comments

Comments
 (0)