Fix #1573: verify delivery reached the terminal before reporting it delivered

[waleedkadous]

Summary

[ok] Message delivered meant only "frames queued on a connected socket" — Tower had no evidence the receiving terminal absorbed anything. This adds the three checks that make the receipt honest: the screen must have settled before we write, the body must be within a stated size limit, and the message's header must actually appear on the terminal — as a result of this write — before the row is marked delivered.

Fixes #1573

Root Cause

Confirmed from the post-#1492 code, not inferred:

• No arrival evidence. submitMessagePaced resolves written when every session.write() returned true, and ShellperClient.write returns true iff the socket object is connected. deliverAgentMail called markDelivered on that alone — no echo, no ACK. A composer that ate the leading bytes (afx send: long architect→builder messages truncated or delivered empty-body #1564) was indistinguishable from a clean delivery.
• No stability requirement in the gate. The render gate proves a composer is empty, never that it has finished being painted. session.lastDataAt was never consulted. The quiescence drain trigger had an accidental 500 ms settle; the request path and the submit fast trigger had none — matching tower: mailbox delivery race eats leading bytes when the write lands mid-composer-settle #1521's "on-demand relays hit it more".
• No size limit anywhere between the CLI flag and the PTY bytes, only the generic 1 MiB HTTP cap and the 48KB --file cap.

Fix

1. Settle-before-write. DeliverySession gains lastDataAt. Delivery requires now − lastDataAt ≥ SETTLE_BEFORE_WRITE_MS (250) both before the per-terminal lock and again inside the precheck, mirroring how the other three write-instant conditions are already double-checked. Written as a positive >= so a session with no usable timestamp (NaN) reads as not settled and holds — a fail-open there would look like a guarantee while being none. One existing test double did in fact lack the field.

2. Loud 48KB limit. MAX_MESSAGE_BYTES + a shared error string in utils/message-format.ts, enforced at the CLI (local, immediate) and at POST /api/send (public route). MAX_FILE_SIZE in both commands/send.ts and commands/reset.ts is now defined as that constant, so the three former copies of 48 * 1024 cannot drift, and the CLI checks after the --file append because attachment content travels in the same body. bodyLength rides every send response that carried a body, through the SDK to afx send's success line. Never a silent truncation.

3. Echo verification. A new required watchEcho port. It is opened immediately before the write and samples how many times the message's header currently appears on the session's gateScreen — the same mirror the gate classifies; the returned EchoWatch.verify() then polls (50 ms, up to 600 ms, full retained buffer including scrollback) until that count is strictly greater.

New evidence, not mere presence, and that distinction is load-bearing: a held attempt leaves its own echo in the same scrollback, so a presence check would match the copy the first attempt left behind and certify a retry whose bytes were swallowed — the false receipt postponed by one redelivery rather than removed. Returning an EchoWatch rather than exposing sample and verify as two ports makes the ordering impossible to get wrong.

A negative means "could not confirm", so the row is held for the existing redelivery machinery: the direction of error becomes a duplicate the agent can see, never a silent loss the sender was told was a success.

Why the needle is normalized

I measured this against live PTYs rather than guessing, because the issue's scope guard asks for change 3 to be cut if it cannot be done reliably.

harness	body	buffer	header found
claude 2.1.252	20 lines	normal	exact form NO, normalized YES, from +200 ms
claude	300 lines / 24.9KB	normal + scrollback	exact YES, buffer line 6, every sample
codex 0.146.0	20 lines	normal	exact YES, every sample
agy	12 lines	alternate	unmeasured (unauthenticated here)

The load-bearing surprise: claude markdown-renders the header on submit. The composer echoes ### [ARCHITECT INSTRUCTION | <ts>] ### verbatim; the transcript then shows [ARCHITECT INSTRUCTION | <ts>], the fences consumed as an H3. An exact-line match would fail on every short claude delivery — the common case. Reducing both sides to their alphanumeric skeleton fixes that and incidentally absorbs quote prefixes, indentation, and wrapping. It stays header-only and harness-agnostic: no screen diffing, no repair, no per-harness branches. The tail is deliberately not used — #1564's message arrived as its final ~30 characters, so a footer needle would have certified the exact corruption this check exists to catch.

Deliberate omissions and residuals

• The issue's optional item 4 (sacrificial leading newline) is not included. writeMessageToSession would send a bare \n as its own write, and whether a harness treats that as "insert newline" or "submit" is exactly the per-harness behaviour I could not measure for codex and agy. An empty submit ahead of every message is a worse failure than the head-eating it guards against, which change 1 already addresses. The issue marks it optional and says to drop it at the first sign of harness weirdness.
• Messages whose normalized first line is under 12 characters skip verification and keep the pre-tower: delivery write edge — verify-or-retry 'delivered', settle-before-write, loud size limit (silent-loss residuals of #1564/#1521) #1573 behaviour. A two-character raw send would match incidental screen text, so confirming it would be a rubber stamp — better to leave it unguarded than to certify it falsely. Verification is therefore not universal: it covers every formatted send (all afx send traffic that carries a header) and raw sends with a distinctive first line.
• agy uses the alternate screen buffer, which has no scrollback, so a message longer than one viewport could scroll its header out of reach and fail to confirm. Consequence is a redelivery, never a dropped message. Documented at the binding; unmeasured because agy is unauthenticated in this environment.
• Nothing bounds redelivery when verification never confirms — filed as tower: bound redelivery when echo verification never confirms (#1573 residual) #1578. Escalation makes such a row visible (markEscalated sets a flag and fires an SSE event) but does not stop the drainer, so the rewrite loop itself is unbounded: one duplicate per clean gate for as long as the harness refuses to show the header. Deliberately not bounded here — every way to bound it means either a new mailbox column or accepting a false delivery after N attempts, and that trade is a decision rather than a bugfix.
• A count false-negative is possible on very long writes: if the write evicts the pre-write copy of the header from the 1000-line mirror, the count comes back equal rather than greater and a genuine delivery reads as unconfirmed. Safe direction (a redelivery), and it compounds the previous point.
• An unconfirmed delivery re-reads and re-normalizes the retained buffer once per poll while the afx send request waits. Bounded by the 600 ms timeout and off the happy path entirely — the measured case confirms on the first read.
• Verification passes on the composer echo as well as the transcript, so a swallowed Enter (typed but never submitted) still verifies. Distinguishing composer from transcript is the classifier's job and is out of scope; the dirty composer holds all following mail, so it surfaces.
• A session that emits output more often than every 250 ms while showing a clean composer would never settle and its mail would starve. Unlikely in practice — output normally makes the composer classify busy, and the existing quiescence trigger already assumes a 500 ms quiet window — and the starvation notice covers visibility.
• Behaviour change worth naming: a --file attachment at or near 48KB plus any message text now fails where it previously went through — for afx send and, because its --file addendum rides inside the generated prompt, for afx refresh too. That is the shared ceiling working as intended, but it is a real change. Both commands check the total body locally (shared messageLimitError), so the refusal is immediate and identically worded rather than a 400 from the route.

Test Plan

• Regression test added — bugfix-1573-delivery-verification.test.ts, 18 tests: the settle window (inside / at the boundary / re-checked in-lock / NaN), the control test (completed write + header never shown → held, not delivered, no broadcast), confirmed delivery with the watch → write → verify ordering pinned, redelivery of a held row, short-needle skip, needle normalization against the three measured rendered forms, and watchEchoOnScreen against a real SessionScreen (composer form, markdown-stripped form, scrolled-into-scrollback, stale header from an earlier attempt, a redelivery that does land, absent, early return). Plus route tests for over-limit / at-limit / bodyLength, and CLI tests for the local refusal, the --file interaction, and the byte-count echo.
• Verified the tests are real: with the settle and verify branches disabled, 6 of 16 fail; with verification reduced to presence-only, the stale-header test fails. Restored, all pass.
• Build passes
• All tests pass — 5344 passed, 48 skipped, 0 failures

Review

CMAP round 1 — gemini APPROVE, claude APPROVE, codex REQUEST_CHANGES: the stale-evidence hole above, which was real and is fixed in d611bb7. Claude's smaller findings (a stray eslint-disable left by a removed debug line, and reset.ts's third 48 * 1024 literal) are fixed in the same commit.

CMAP round 2 — gemini APPROVE, codex APPROVE (flipped, no remaining issues), claude COMMENT with five findings, all addressed: the unbounded-redelivery residual is filed as #1578; afx refresh's route-400 regression is fixed with a local precheck; the stale {@link DeliveryPorts.verifyEcho} doc link and the test count in this body are corrected; the count false-negative and the per-poll re-normalization are documented as residuals here and at the binding.

tower-routes.test.ts's gateSession double now echoes writes into its own mirror, because a real terminal does and the delivery path now depends on it. The #1492 suite stays green.

Rebased onto #1575 (self-attesting frames, #1574)

#1575 landed first and touches message-format.ts and tower-routes.ts. Both conflicts were additive-vs-additive and keep both sides.

The new frame composes with echo verification rather than breaking it, and strengthens it. Headers now read ### [ARCHITECT INSTRUCTION → <toAgent> | <ts>] ###; the needle is derived from the row's actual formatted_message, so it picked the new shape up for free. The arrow normalizes away as punctuation while the recipient name stays in the needle — so verification is now recipient-specific: a frame delivered to the wrong agent cannot satisfy the right agent's check. The frame is still 3 lines, so #1574's own constraint about PACED_WRITE_LINE_THRESHOLD holds and the write path this PR guards is byte-identical.

Fixtures now build the frame with formatArchitectToBuilderMessage rather than hand-writing it, plus an explicit assertion that the recipient segment reaches the needle. That is the durable lesson from the rebase: a hand-copied header would have gone on passing while the frame it modelled drifted away from what the delivery path actually writes.

Post-merge: 5360 passed, 48 skipped, 0 failures (including #1574's 16 new tests).

[waleedkadous]

Architect review (posted as comment — GitHub blocks a formal review from the PR author's own account): APPROVED and admin-merged on the owner's explicit word. Matches the #1573 prescription with three reviewed-and-accepted deviations (sacrificial newline dropped per the issue's escape hatch; unbounded-redelivery residual escalated as #1578; shared 48KB ceiling's --file behavior change accepted). The occurrence-count echo-verification (new evidence, not presence) is the right answer to codex's stale-echo finding, and measuring real harness rendering before shipping the matcher was exactly the discipline this write edge needed. CI green on 416eaf3 post-rebase.