Skip to content

Aggregated page-session card: one card per run of page actions #413

Description

@omridevk

Parent

Related: #344 (tool-cards epic — this builds on its merged card families + element-capture pipeline). Not a sub-issue; new scope.

What to build

A turn where the AI drives the live page currently renders as a flat stack of per-action tool cards (Typed ×3, Selected, Checked a box ×4, …). Instead, consecutive conciv_page acts should render as ONE aggregated page-session card that tells the story of what was done on the page:

  • Streaming: the card appears on the first act call, auto-open — steps append live, the active step shimmers, an "acting" badge shows in the header. No screen area while driving (the real page behind the widget is the live view).
  • Settled collapsed: one summary line — status dot + Edited the page · N actions.
  • Settled expanded: browser-chrome header + step rail (glyph · verb · target · value per step, done/error states), step targets labeled from the element-capture descriptors (accessibleName), so history reads like a story, including for old sessions.

Approved visual reference (spike with real screenshots; exhibit C = streaming, exhibit A minus screen = this ticket's settled card): https://claude.ai/code/artifact/f56d1a79-8e4b-4060-b884-f1bfcef92e97 (private artifact — ask Omri for access).

Out of scope (follow-up ticket, split later): the card's screen area — poster, midscene-style scripted replay (camera pan/zoom + pointer over per-step visuals), and rrweb live replay. This ticket is the transcript-only card. Design note for that follow-up: with element captures db-persisted, per-step ElementPreview replicas may beat keyframe stills.

Acceptance criteria

  • A turn of consecutive page acts renders exactly one page-session card in the thread; the old per-action cards do not render for those calls.
  • Reply text, any non-page tool call, or turn end closes the group; interleaved page reads and blank text do NOT split it.
  • Tool-result parts of the session's calls fold into the segment (a real transcript [act, result, act] stays one card).
  • Streaming behavior: card open while running with active-step shimmer; works in both thread and activity views (activity's live-segment selection updated for the new segment kind).
  • Step targets prefer capture accessibleName, falling back to input selector/name/ref; targetless verbs (css, eval, effect) get fixed labels.
  • Consumers that don't opt in (generic message primitive) keep today's flat grouping — zero behavior change without the new options.
  • Old sessions (transcript reload) render the aggregated card from history alone.
  • Storybook stories (Streaming / Settled / Expanded / WithError) + real-browser tests per repo testing rules; embed rebuilt and widget IT green.

Blocked by

None — can start immediately (the tool-cards stack #348/#399 merged 2026-08-10; that was the original gate).


Design decisions (settled)

Question Decision
Step source conciv_page act tool calls (transcript truth), NOT the recorder's rrweb distilled log
Retention Step data from the epic's db-persisted element captures; rrweb never a rendering dependency
Collapsed view Summary line (mini-browser poster arrives with the replay follow-up)
Streaming view Auto-open ledger, no screen area
Group bounds Consecutive page.* mutating calls; reads fold silently; reply text / non-page tool / turn end closes
Plumbing Client-side grouping + injected renderer; no server aggregation, no new wire concept

References studied: assistant-ui computer-use element, alibaba/page-agent (SimulatorMask, act-time rects), midscene visualizer (scene-script replay model — adopted for the follow-up ticket), BrowserTrace, Skyvern, Playwright trace viewer.


Implementation plan (revision 4 — targets post-epic main)

Grounded in the merged tool-cards stack. Before starting: re-verify every cited path/symbol against current main — they were read from the pre-merge kit branch on 2026-08-10.

Two hardening review rounds (codex) are baked in: result-part folding, blank-text folding, verb parsing over input before arguments, renderer contract owned by ui-kit-chat, injection via Thread.Messages/Activity.Root props (never ToolViewCtx), activity last-renderable-segment fix, retention-asserting fixtures with JSON-string result content.

Global constraints

  • Functions only, no classes, no IIFEs, ZERO code comments. oxfmt style. No any/as/@ts-ignore.
  • solid-primitives over raw signals; splitProps, never destructure props.
  • Kit reuse mandate: compose CardShell/cardHeader, Chip/ChipRow, StatusVisual, CollapsibleSection, ErrorBlock, ElementPreview from @conciv/ui-kit-chat/tools — no hand-rolled card chrome; conciv/tool-card-shell lint boundary stays green.
  • Real-browser tests, native assertions, no test-ids, no DOM-measurement assertions. Story iteration against running storybook; full suites once at the end.
  • Gates per package with bare turbo run test --filter=<pkg>; fallow audit before commit; commit with pathspec.

Task 1: page-session segment kind in grouping

packages/ui-kit-chat/src/store/grouping.ts + its test file.

export type PageSessionSegment = {kind: 'page-session'; indices: number[]}
export type Segment = ChainSegment | ReplySegment | PageSessionSegment
export type GroupingOptions = {pageActVerbs?: ReadonlySet<string>; pageToolName?: string}
export function groupSegments(parts: ReadonlyArray<MessagePart>, options?: GroupingOptions): Segment[]
export function pageSessionVerbOf(part: MessagePart, toolName?: string): string | null

Segmentation rules, in precedence order over the existing reduce:

  1. Non-empty text part → reply (unchanged; closes any open segment).
  2. conciv_page tool-call whose verb is in pageActVerbs → extends an open page-session or opens one.
  3. Folds into an OPEN session without opening one: a conciv_page call with a verb outside the set (reads); a tool-result whose toolCallId matches a conciv_page call already in the session; a blank text part.
  4. Anything else → existing chain behavior (closes the session).
  5. No options → output byte-identical to today.

pageSessionVerbOf is total over BOTH part representations, part.input first then part.arguments JSON (mirroring parseInput), so input-streaming calls with structured input open the session on the first act. Track open-session call ids in a local Set for O(1) result folding.

Tests (failing first; adapt fixture field names to the repo's @tanstack/ai-client types):

  • acts + paired results fold into one session
  • interleaved reads and blank text fold without opening a session
  • a lone read or a foreign result stays chain
  • reply text and non-page tools close the session ([act, text, bash, act] → session/reply/chain/session)
  • a streaming call with structured input opens the session
  • without options nothing changes (existing suite untouched)

Task 2: Renderer contract + act-verb source

New packages/ui-kit-chat/src/store/page-session.ts (types OWNED by ui-kit-chat so consumers depend inward; re-export from the package index):

export type PageSessionRenderProps = {
  parts: ReadonlyArray<ToolCallPart>
  resultFor: (toolCallId: string) => ToolResultPart | undefined
  actVerbs: ReadonlySet<string>
  streaming: boolean
}
export type PageSessionRenderer = (props: PageSessionRenderProps) => JSX.Element
export type PageSessionConfig = {render: PageSessionRenderer; actVerbs: ReadonlySet<string>}

In packages/extensions/page/src/shared/defs.ts (pure data, no client imports):

export const PAGE_ACT_VERBS: ReadonlySet<string> = new Set(
  PAGE_TOOL_DEFS.filter((def) => def.meta.mutating === true).map((def) => pageVerbOfTool(def.name)),
)

Guard test: click/fill in, snapshot/route out.

Task 3: SessionCard in the page extension card family

New packages/extensions/page/src/client/cards/session-card.tsx (+ stories + browser test), beside the merged family cards (act-card, edit-live-card, …), composed from kit primitives and the family's shared.tsx helpers.

export type PageSessionStep = {verb: string; target: string; value?: string; state: 'streaming' | 'complete' | 'error'}
export function SessionCard(props: PageSessionRenderProps): JSX.Element
export function pageSessionSteps(
  parts: ReadonlyArray<ToolCallPart>,
  resultFor: (toolCallId: string) => ToolResultPart | undefined,
  captureFor: (toolCallId: string) => ToolCaptureView | undefined,
  actVerbs: ReadonlySet<string>,
): PageSessionStep[]

pageSessionSteps is pure. Per act call: target = capture descriptor accessibleName (after first, else before) → fallback input selector/name/ref; value = input value → else descriptor value; targetless verbs get fixed targets (stylesheet, script, effect); state from the paired result (error/missing-while-streaming/complete). Reads drop. Captures come from the same ctx mechanism the family cards use (captureFor(toolCallId) — the session card spans many calls, so the accessor, not props.capture).

Anatomy: header (page glyph, Edited the page, N actions subtitle, family mutatingBadge, kit status vocabulary for the live "acting" state) · step rail in a CollapsibleSection (open = streaming, click pins) · failed steps mark rows; last-result error shows ErrorBlock.

Stories: Streaming / Settled / Expanded / WithError, reusing the family's story fixtures and the kit's element-capture.fixtures.ts. Result fixtures use JSON-STRING content (that is the repo's result representation).

Task 4: Thread/activity injection + app wiring

  • Thread.Messages gains optional pageSession?: PageSessionConfig, stored in ThreadConfigContext; segment <Switch> gains an asPageSession arm beside asChain/asReplyresultFor = (id) => pairing().byCallId.get(id), streaming mirrors the chain arm's last-segment computation.
  • Activity.Root gains the same optional prop into ActivityConfigContext; resultFor = the view's reactive activity.resultFor; ONE shared segment→props helper so the two surfaces cannot drift. Replace the last-visible-chain live-segment calculation with last-renderable-segment (chain OR page-session) + an activity-level streaming test.
  • Do NOT extend ToolViewCtx (protocol-level runtime tool context). The generic primitives/message/message.tsx groupSegments call site stays untouched — no options, no behavior change (recorded decision).
  • App wiring in apps/conciv chat pane: pass {render: SessionCard, actVerbs: PAGE_ACT_VERBS} to both surfaces (the app already wires ToolProvider + tools there).
  • Thread test: [page fill, result, page fill, result, text reply] → exactly one Edited the page card + reply prose, per-action cards absent. Then embed rebuild + widget IT.

Task 5: Repo gates

pnpm typecheck, pnpm test:affected (whiteboard suite stays CI-only), fallow audit zero INTRODUCED.


🤖 Generated with Claude Code


Approved mockup (visual source of truth)

Durable source (preferred): docs/assets/page-session-card-spike/ — committed via PR #414. Regenerate the mockup in ~30s: node docs/assets/page-session-card-spike/capture.mjs && node docs/assets/page-session-card-spike/build-spike.mjs, then open the generated page-session-card-spike.html. Screenshots are produced on demand, never stored in git.

Artifact (original): https://claude.ai/code/artifact/f56d1a79-8e4b-4060-b884-f1bfcef92e97
(Claude artifact, private to Omri's account and may orphan on account/org switches — hence the committed copy. Version label real-screenshots; the earlier first-cut version used live DOM and is superseded.)

This is the design-approved look for the card, validated in two rounds on 2026-08-10. Implementations should match its proportions, hierarchy, and pacing — deviations need a screenshot comparison in review.

What the page shows

Three exhibits, top to bottom:

  • Exhibit A — expanded card with replay running. Browser chrome (traffic dots + mono URL pill localhost:3000/form + blue "replay" badge), a 16:9.6 screen, a status bar (verb chip · target · n/m counter · play button), and the step rail (7 rows: fill/fill/fill/select/check/check/fill with quoted values). The replay: camera pans/zooms onto each acted element, a pointer eases between targets with a click ripple, the screenshot crossfades to the post-action frame, the rail and status bar stay in sync; clicking a step row seeks. For THIS ticket only the non-screen parts apply (chrome, status, rail, states) — screen/replay is the follow-up ticket.
  • Exhibit B — collapsed, settled. Chrome + poster (final frame with the action-trail dots at ramping opacity) + one summary line: green status dot · "Filled the profile form" · 7 actions · 12s · chevron. This ticket ships the summary-line form; the poster joins in the follow-up.
  • Exhibit C — streaming. THE reference for this ticket's live state: chrome with pulsing "acting" badge, no screen, step rows appending (✓ done rows with values, active row with shimmer "Checking 'Accept terms'…").

How it was built (why it's trustworthy)

The screen content is NOT drawn UI: a fake profile-form page was driven in headless Chromium via Playwright — each field filled/checked exactly as the page tools would — and screenshotted after every action (8 JPEG frames @2x DPR, ~612KB total), with each target element's bounding box recorded. The replay is those real screenshots swapped under a CSS-transform camera. That is byte-for-byte the data shape production has (per-step visuals + element rects + timestamps), so the mockup is the honest floor of what ships, not an aspirational render.

Validated motion values (for the follow-up replay ticket)

  • Camera + pointer travel ~680ms, cubic-bezier(0.32, 0.72, 0, 1) camera / (0.4, 0, 0.2, 1) pointer
  • Click ripple ~240ms; frame crossfade ~260ms; post-action dwell ~620ms
  • Trail dots: last ≤3 steps, opacity ramp; prefers-reduced-motion: no transitions, discrete jumps

Regenerating the mockup

The artifact is self-contained (screenshots embedded as data URIs). It was generated by three throwaway scripts (fake-form.htmlcapture.mjs (Playwright, 800×520 @2x) → build-spike.mjs) in the drafting session's scratchpad — ephemeral; if lost, the pipeline is trivial to recreate from this description.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentTicket is fully specified and agent-grabbable

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions