Skip to content

Commit 3b61849

Browse files
committed
optical-codec: read eggs, and set the band alphabet from measurement
The reader finds a stack by its caps, derives the band height from them (a cap is one band tall, so an egg is self-describing at any capture scale), corrects colour affinely against the sampled caps, and refuses rather than guesses when a band's nearest colour is not clearly nearest. Two things the sweep caught, both of which changed the design: Grain makes every row a colour change, so finding the band count from transitions worked and then read nothing at all under noise. A band is one colour across its width, so a median filter across the width removes the noise and cannot move a horizontal boundary. A band whose luma is near a cap's gets read as a cap, and the egg then vanishes entirely rather than being misread. Seven of sixteen codes were being lost this way. Palettes are now packed by script under a luma bound, which cost almost nothing in separation. Six pixels a band reads twelve colours through the whole degradation suite. Octal ships, since the suite is synthetic and the game's own rendering is not in it.
1 parent 0d60401 commit 3b61849

9 files changed

Lines changed: 1155 additions & 160 deletions

File tree

‎apps/screen-angel/EGGS.yml‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
meta:
99
id: eggs
1010
status: design
11-
built: []
11+
built: [band_code, egg_renderer, egg_reader, measured_band_height]
1212
not_built: [egg_object, menu_tree_compile, selection_accumulator, hatch_semantics, resume_point, recall]
1313
see_also: [OPTICAL-CHANNEL.yml, ANGEL-EVENT-BUS.yml, RECORDER.yml, modules/soul-angel/OUT-OF-GAME-JOBS.yml, modules/soul-angel/UNIVERSAL-JOBS.yml]
1414
upstream: "moollm skills/soul-city/GLOSSARY.md (Egg, Errand) and skills/soul-city/SOUL-BRIDGES.md (The errand)"
@@ -224,6 +224,14 @@ the_blip_editor:
224224
legible_to_both_readers: "A player learns 'gold cap band means news' in one session, and the reader decodes the same stripes numerically. One artifact, two audiences, which is the property the whole optical channel rests on (OPTICAL-CHANNEL.yml, the_player_reads_it_too)."
225225
authored_once: "Shape, bands, caps, palettes, states and the DGRP table are emitted by the same compile that emits the menu tree and the verb registry (compiled_from_one_source)."
226226

227+
the_numbers_are_measured_now:
228+
where: "packages/optical-codec — egg-render.ts draws them, egg-read.ts reads them, scripts/measure-eggs.ts sweeps band height against the degradation suite, and ../RECOGNIZER.yml, the_egg_reader has the full grid."
229+
a_band_is_six_pixels: "Six device pixels a band reads twelve colours through everything in the suite: a window at a third scale, soft focus, a warm noisy phone grab, a tilted one. Five is enough for all but the smallest window. That is the number the art has to hit at the zoom where a code is meant to be read, and it is a drawing requirement, not a hope."
230+
the_caps_earn_their_place: "Everything the section above claimed for them holds up under measurement, and two of the three turned out to be load-bearing in a way that was easy to underrate. A cap is exactly one band tall, so the reader derives the band height from the picture instead of being told it — an egg is self-describing at any zoom or capture scale. And correcting the bands by the sampled cap colours means an evening-light frame reads exactly as well as a clean one, which is the single best cell in the grid."
231+
what_the_bands_may_not_be: "A colour whose luma is near either cap's gets read AS a cap, and then the egg is not misread, it is invisible. So the palette is bounded away from both — luma 72 to 188 — and the palettes are packed by a script that maximises the smallest gap under that bound rather than chosen by eye (src/egg-code.ts)."
232+
twelve_is_legible_and_eight_is_shipped: "The measurement says twelve colours a band survives the suite. The default is octal anyway: the suite is synthetic and the game's own rendering is not in it, and an octal band is the same symbol as a dialled compass point (cracking_it_like_a_safe). The version band is the second thing on the stack precisely so this can change later."
233+
what_is_still_not_known: "Whether any of it survives the game actually drawing an egg. Nothing here has been checked against a frame the game produced, because there is no egg in the game yet."
234+
227235
# WHAT AN EGG IS, IN OTHER PEOPLE'S VOCABULARIES
228236
#
229237
# Several fields already named this object. Each name is worth having because each one carries an

‎apps/screen-angel/RECOGNIZER.yml‎

Lines changed: 26 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,8 @@
88
# MOBILE-CAMERA.yml (the hard optical path) · RECORDER.yml (where frames come from)
99

1010
status:
11-
built: [codec_package, qr_path, synthetic_scene_renderer, egg_renderer, abuse_suite_minus_jpeg, recognize_scan_verb]
12-
not_built: [frame_source, gpu_prefilter, egg_path, worker_pool, overlay_debug_view]
11+
built: [codec_package, qr_path, synthetic_scene_renderer, egg_renderer, egg_reader, band_alphabets, abuse_suite_minus_jpeg, recognize_scan_verb, coverage_font_reader, anchor_matcher]
12+
not_built: [frame_source, egg_path_in_the_verb, gpu_prefilter, worker_pool, overlay_debug_view, verification_against_a_real_frame]
1313
decided: [zxing_for_qr, local_only_no_remote_model, codec_first_in_node]
1414
where: "packages/optical-codec (codec, renderer, fixtures, tests) · src/main/recognize/service.ts (the verb)"
1515

@@ -130,11 +130,33 @@ measured_legibility:
130130
what_it_decides: "How big the About dialog draws its code, and what to say to anyone who wants it smaller."
131131

132132
the_eggs_are_drawn_before_the_game_draws_them:
133-
what: "packages/optical-codec renders eggs into synthetic frames — caps, calibration swatches, bands, segments, levitation, glow — and reports the ground-truth box and digits for each one."
133+
what: "packages/optical-codec renders eggs into synthetic frames — caps, bands, segments, levitation, glow — and reports the ground-truth box and digits for each one."
134134
why_not_wait_for_screenshots: "There are no eggs in The Sims yet, so there is nothing to photograph. Generating the frame gives the reader targets today and stays useful afterwards: a real capture tells you whether it works, a synthetic frame tells you which pixel broke it."
135135
the_test_that_matters: "The bands are pixel-identical lit and dark. The glow draws behind the body, so the cosmetic layer cannot reach the pixels the reader samples — which is the rule above, enforced rather than asserted."
136136

137+
the_egg_reader:
138+
where: "packages/optical-codec/src/egg-read.ts, swept by scripts/measure-eggs.ts."
139+
how_it_finds_one: "Column by column: the brightest and darkest runs in a column are candidate caps, and a candidate survives only if the distance between their centres divides evenly by a band height that also explains the colour changes between them. Then adjacent columns that agree are grouped, and a group narrower than a band is tall is thrown away."
140+
the_caps_do_three_jobs: "They delimit the stack, they set the scale — a cap is exactly one band tall, so the metric comes out of the picture rather than out of a config file — and they are the white and black reference for an affine colour correction. A sunset shifts the bands and the caps by the same amount, and correcting by the caps removes it before a single digit is decided."
141+
it_refuses_rather_than_guesses: "Every band reports the distance to the runner-up colour. The smallest of those is the reading's confidence, and a reading below the floor is returned as refused WITH its digits, so a caller can see what was nearly there. A failed check digit is refused the same way."
142+
the_measured_floor: "SIX PIXELS A BAND reads twelve colours through every degradation in the suite. Five reads twelve through all but a window at a third scale. Below four, only a clean or evenly-dimmed grab reads at all."
143+
the_grid:
144+
clean_screen_grab: "2px and up, twelve colours."
145+
evening_light: "2px and up. A global dim costs nothing, because the caps dim with the bands and the correction is affine."
146+
window_at_50_percent: "4px and up."
147+
window_at_33_percent: "6px and up."
148+
soft_focus: "5px and up."
149+
phone_noise_and_warm_cast: "2px and up, once the frame is median-filtered across the width."
150+
phone_noisy_and_soft: "5px and up."
151+
phone_tilted: "4px and up."
152+
two_findings_worth_keeping:
153+
noise_invents_boundaries: "Finding the band count from colour changes works and then stops working the moment there is grain, because every row becomes a change. A band is one colour all the way across, so a median of five neighbours across the width removes it and cannot move a horizontal boundary. Without that filter every noisy cell in the sweep read nothing at all."
154+
a_band_can_be_mistaken_for_a_cap: "The reader looks for the caps as the extremes of a column's range and claims the top and bottom 15% of it, so a dark band inside that claim becomes a black cap and the whole egg vanishes — not misread, not refused, invisible. Seven of sixteen codes were being lost this way. The fix is a bound on the palette, not on the reader: bands live between luma 72 and 188 (src/egg-code.ts)."
155+
what_the_alphabet_is_set_to: "Octal, with twelve measured as legible. The headroom is spent on margin rather than on digits: the sweep is synthetic, the game's own rendering is not in it, and an octal band is the same symbol as a dialled compass digit (EGGS.yml, cracking_it_like_a_safe)."
156+
still_only_pixels_we_drew: "Same caveat as the QR numbers. tools/verify-frame.ts is the thing that turns this from a floor into a fact, and it needs one real frame."
157+
137158
open_items:
138159
packaging_the_wasm: "The Node path finds the wasm inside node_modules by walking up from an exported entry point. A packaged app needs those two files unpacked from the asar and resolved from resources instead."
139160
jpeg_in_the_abuse_suite: "Needs a real DCT and quantisation table. Standing in for it with blur plus noise would let us claim a pass we had not earned."
140-
the_palette_is_provisional: "Ten separated RGB values, chosen against theory. The spec requires choosing them against the game's rendered output, and that has not happened."
161+
the_palette_is_provisional: "The palettes are packed rather than picked — scripts/pick-palettes.ts maximises the smallest gap between colours, under a distance from the caps and a luma window. What is settled is HOW MANY colours a band can hold and how far apart they have to be. Choosing them against the game's own rendered output has not happened, and the sweep cannot do it."
162+
the_egg_field_is_still_empty: "The reader exists in the codec and the verb still returns an empty array. Wiring is small; doing it before there is a frame to verify against would mean shipping a path nobody has seen work."
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
/**
2+
* How many colours can a band carry, and how few pixels tall can a band be?
3+
*
4+
* The spec said "ten to twelve maximally separated colours, chosen against the game's actual rendered
5+
* output rather than against theory". This is the theory-free half: sweep every alphabet against every
6+
* band height under every degradation, and let the grid say which pairs survive. The remaining half
7+
* needs a real screenshot of the real sprites, and no amount of this substitutes for it.
8+
*
9+
* A cell passes only if EVERY code in the sample decodes exactly. One misread in twenty is not a
10+
* legible cell — it is a cell that will be wrong in front of a player.
11+
*
12+
* pnpm --filter @micropolis/optical-codec tsx scripts/measure-eggs.ts
13+
*/
14+
15+
import { blur, noise, perspective, roundTrip, tint } from '../src/degrade';
16+
import { ALPHABETS, fieldLimits, separation, type BandAlphabet, type EggCode } from '../src/egg-code';
17+
import { readEggs } from '../src/egg-read';
18+
import { drawEgg } from '../src/egg-render';
19+
import { createRaster, fillRect, type Raster } from '../src/raster';
20+
21+
const BAND_HEIGHTS = [2, 3, 4, 5, 6, 8];
22+
const SAMPLE = 16;
23+
24+
const conditions: { name: string; apply: (r: Raster) => Raster }[] = [
25+
{ name: 'clean screen grab', apply: (r) => r },
26+
{ name: 'window at 50%', apply: (r) => roundTrip(r, 0.5) },
27+
{ name: 'window at 33%', apply: (r) => roundTrip(r, 0.34) },
28+
{ name: 'soft focus', apply: (r) => blur(r, 1) },
29+
{ name: 'evening light', apply: (r) => tint(r, [0.62, 0.7, 1.05]) },
30+
{ name: 'phone: noise + warm cast', apply: (r) => noise(tint(r, [1.1, 1, 0.82]), 18) },
31+
{ name: 'phone: noisy and soft', apply: (r) => noise(blur(r, 1), 12) },
32+
{
33+
name: 'phone: tilted',
34+
apply: (r) =>
35+
perspective(r, {
36+
tl: [8, 4],
37+
tr: [r.width - 3, 13],
38+
br: [r.width - 10, r.height - 6],
39+
bl: [3, r.height - 12]
40+
})
41+
}
42+
];
43+
44+
/** A deterministic spread of codes, so a failing cell can be reproduced exactly. */
45+
function codes(alphabet: BandAlphabet): EggCode[] {
46+
const base = alphabet.colours.length;
47+
const limits = fieldLimits(alphabet);
48+
const out: EggCode[] = [];
49+
let state = 20260917;
50+
// High bits, not low. An LCG's low bits barely move, and `state % base` on a small base returned
51+
// the same digit sixteen times — a sample of one code wearing a sample of sixteen's clothes.
52+
const next = (n: number): number => {
53+
state = (state * 1103515245 + 12345) & 0x7fffffff;
54+
return Math.floor((state / 0x80000000) * n);
55+
};
56+
for (let i = 0; i < SAMPLE; i++) {
57+
out.push({ kind: next(base), version: 1 % base, id: next(limits.id + 1), value: next(limits.value + 1) });
58+
}
59+
return out;
60+
}
61+
62+
/** One egg on a floor, with the room furniture the reader has to ignore. */
63+
function scene(code: EggCode, alphabet: BandAlphabet, bandHeight: number): Raster {
64+
const radius = Math.max(4, Math.round(bandHeight * 1.6));
65+
const width = radius * 6;
66+
const height = bandHeight * 12 + 24;
67+
const frame = createRaster(width, height, [92, 84, 72]);
68+
fillRect(frame, 0, 0, width, Math.round(height * 0.35), [64, 72, 96]);
69+
drawEgg(frame, code, { x: Math.round(width / 2), y: height - 8 }, { alphabet, bandHeight, radius, lit: 'steady' });
70+
return frame;
71+
}
72+
73+
function passes(alphabet: BandAlphabet, bandHeight: number, apply: (r: Raster) => Raster): boolean {
74+
for (const code of codes(alphabet)) {
75+
const frame = apply(scene(code, alphabet, bandHeight));
76+
const readings = readEggs(frame, { alphabet });
77+
const got = readings.find((r) => r.code !== null)?.code;
78+
if (got === undefined) return false;
79+
if (got.kind !== code.kind || got.version !== code.version || got.id !== code.id || got.value !== code.value) return false;
80+
}
81+
return true;
82+
}
83+
84+
console.log(`${SAMPLE} codes a cell, all of which must decode exactly. Cell shows the largest alphabet that did.`);
85+
console.log('');
86+
console.log(`${'band height, pixels'.padEnd(28)}${BAND_HEIGHTS.map((h) => String(h).padStart(6)).join('')}`);
87+
88+
for (const condition of conditions) {
89+
const cells = BAND_HEIGHTS.map((bandHeight) => {
90+
let largest = 0;
91+
for (const alphabet of ALPHABETS) {
92+
if (passes(alphabet, bandHeight, condition.apply)) largest = Math.max(largest, alphabet.colours.length);
93+
}
94+
return (largest === 0 ? '--' : String(largest)).padStart(6);
95+
});
96+
console.log(`${condition.name.padEnd(28)}${cells.join('')}`);
97+
}
98+
99+
console.log('');
100+
console.log('closest pair in each alphabet, which is the noise budget a band lives inside:');
101+
for (const alphabet of ALPHABETS) {
102+
console.log(` ${alphabet.name.padEnd(12)} ${alphabet.colours.length} colours ${separation(alphabet).toFixed(1)}`);
103+
}
Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
/**
2+
* Choosing the band palettes, instead of picking colours that sound well separated.
3+
*
4+
* Hand-picked palettes had a defect the sweep exposed: octal and decimal came out with the SAME
5+
* closest pair, because orange sits between red and yellow and both palettes held all three. An
6+
* alphabet that buys two more digits per band for no loss in separation is not a trade-off, it is a
7+
* sign the smaller alphabet was badly chosen — so the smaller ones are now farthest-point selections
8+
* from a pool, and the ladder means something.
9+
*
10+
* The pool is constrained by what the READER needs, not by taste:
11+
*
12+
* away from the caps Nothing within 60 of white or black, which are the delimiters and the
13+
* colour references. A band that looks like a cap destroys the structure the
14+
* check digit is computed over.
15+
* inside the thresholds Luma between 72 and 188. The cap finder claims the top and bottom 15% of a
16+
* column's range, so a band has to stay clear of both with margin to spare.
17+
* Measured, not guessed: at 46 a dark blue band sat at 15.1% of the range and
18+
* was read as a black cap, and seven of sixteen codes then failed to be found
19+
* at all under blur plus noise.
20+
* saturated or grey A colour has to be far from its neighbours after a colour cast, and the
21+
* chromatic ones survive that best. One grey is allowed and is useful: it is
22+
* far from every hue.
23+
*
24+
* pnpm --filter @micropolis/optical-codec tsx scripts/pick-palettes.ts
25+
*/
26+
27+
type RGB = [number, number, number];
28+
29+
const CAP_MARGIN = 60;
30+
const LUMA_LOW = 72;
31+
const LUMA_HIGH = 188;
32+
33+
const luma = (c: RGB): number => 0.299 * c[0] + 0.587 * c[1] + 0.114 * c[2];
34+
const dist = (a: RGB, b: RGB): number => Math.hypot(a[0] - b[0], a[1] - b[1], a[2] - b[2]);
35+
36+
/** Every colour on a coarse RGB lattice that a band is allowed to be. */
37+
function pool(): RGB[] {
38+
const out: RGB[] = [];
39+
for (let r = 0; r <= 255; r += 8) {
40+
for (let g = 0; g <= 255; g += 8) {
41+
for (let b = 0; b <= 255; b += 8) {
42+
const c: RGB = [r, g, b];
43+
const l = luma(c);
44+
if (l < LUMA_LOW || l > LUMA_HIGH) continue;
45+
if (dist(c, [255, 255, 255]) < CAP_MARGIN || dist(c, [0, 0, 0]) < CAP_MARGIN) continue;
46+
out.push(c);
47+
}
48+
}
49+
}
50+
return out;
51+
}
52+
53+
function minPair(set: RGB[]): number {
54+
let worst = Number.POSITIVE_INFINITY;
55+
for (let i = 0; i < set.length; i++) {
56+
for (let j = i + 1; j < set.length; j++) worst = Math.min(worst, dist(set[i], set[j]));
57+
}
58+
return worst;
59+
}
60+
61+
/**
62+
* Farthest-point insertion, then hill-climb by swapping the member of the closest pair.
63+
*
64+
* Maximising the smallest gap in a set is a packing problem nobody solves exactly for this size. The
65+
* greedy start plus swaps gets within a few units of the best known packings, and the number it
66+
* reports is measured either way — which is all the sweep needs from it.
67+
*/
68+
function pick(candidates: RGB[], k: number, seed: RGB): RGB[] {
69+
const chosen: RGB[] = [seed];
70+
while (chosen.length < k) {
71+
let best: RGB = candidates[0];
72+
let bestGap = -1;
73+
for (const c of candidates) {
74+
const gap = Math.min(...chosen.map((s) => dist(c, s)));
75+
if (gap > bestGap) {
76+
bestGap = gap;
77+
best = c;
78+
}
79+
}
80+
chosen.push(best);
81+
}
82+
83+
for (let round = 0; round < 400; round++) {
84+
let improved = false;
85+
for (let i = 0; i < chosen.length; i++) {
86+
const without = chosen.filter((_, j) => j !== i);
87+
let best = chosen[i];
88+
let bestGap = Math.min(...without.map((s) => dist(chosen[i], s)));
89+
for (const c of candidates) {
90+
const gap = Math.min(...without.map((s) => dist(c, s)));
91+
if (gap > bestGap) {
92+
bestGap = gap;
93+
best = c;
94+
}
95+
}
96+
if (best !== chosen[i]) {
97+
chosen[i] = best;
98+
improved = true;
99+
}
100+
}
101+
if (!improved) break;
102+
}
103+
return chosen;
104+
}
105+
106+
/** Top to bottom by hue, so a person reading a stack sees a wheel rather than a scramble. */
107+
function byHue(set: RGB[]): RGB[] {
108+
const hue = (c: RGB): number => {
109+
const max = Math.max(...c);
110+
const min = Math.min(...c);
111+
if (max === min) return -1; // grey first, since it has no hue to sort by
112+
const d = max - min;
113+
const h = max === c[0] ? ((c[1] - c[2]) / d + 6) % 6 : max === c[1] ? (c[2] - c[0]) / d + 2 : (c[0] - c[1]) / d + 4;
114+
return h * 60;
115+
};
116+
return [...set].sort((a, b) => hue(a) - hue(b));
117+
}
118+
119+
const candidates = pool();
120+
console.log(`${candidates.length} colours in the pool: luma ${LUMA_LOW}-${LUMA_HIGH}, at least ${CAP_MARGIN} from either cap.`);
121+
console.log('');
122+
123+
for (const k of [4, 6, 8, 10, 12]) {
124+
const set = byHue(pick(candidates, k, [208, 48, 48]));
125+
console.log(`${k} colours, closest pair ${minPair(set).toFixed(1)}:`);
126+
for (const c of set) console.log(`\t\t[${c[0]}, ${c[1]}, ${c[2]}], // luma ${luma(c).toFixed(0)}`);
127+
console.log('');
128+
}

0 commit comments

Comments
 (0)