You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Add a /study command that records what surprised a piece of work — a mid-flight redesign, a reviewer finding that had to be actioned, an acceptance criterion that turned out ambiguous — as a short, classified table in docs/study/<issue>.md. Run at any point, it complements /build and /verify without being wired into either. Over time the accumulated files answer a question no single PR can: which part of the harness keeps costing us, and therefore what to fix.
Motivation
/build and /verify now carry a piece of work from a GitHub issue to a reviewed, test-green branch. What neither leaves behind is any record of where the work went sideways on the way. A reviewer finding gets fixed and disappears into a commit; an ambiguous criterion gets interpreted and never mentioned again. Each is invisible individually, and collectively they are the only evidence of whether the harness itself needs work.
A sister project running the same command set under an autonomous dispatcher has been keeping exactly this record. Fifty-nine such files accumulated there in about two weeks, and the tally across them reads 142 findings at Execution level and 60 at Plan-spec — a clear, actionable reading (the doing is lossy; the specs are mostly sound) that no individual PR could have produced. NetPace ships far more slowly, so the aggregate will take much longer to become legible here. That is an argument for starting the record cheaply now, not for waiting: files never written can never be read back.
Three design points follow from that project's experience:
A clean run must write nothing. With no "nothing notable" entry, the presence of a file means the work taught us something, and the folder stays all-signal. The denominator — how many features shipped without incident — stays recoverable from merged-PR history.
A level nobody can populate is worse than no level. That project also carried a Goal level, for work whose aim was itself misaimed. It recorded zero in two weeks, because whether an issue should have existed at all is settled when the issue is drafted, not downstream while building it. That judgement is not available to the thing doing the work, so the level sat empty and diluted the tally. It is deliberately absent here.
A surprise is not always our process's fault. Execution and Plan-spec both locate the fault in how we work — the doing, or the drafting. Neither covers a surprise that came from the code as it already was, or from something outside the repository entirely. Those get their own levels, because their fixes have different owners.
Users & jobs
Someone who has just finished a piece of work. They ran /build and /verify, something during the run was unexpected, and they want it recorded before the context evaporates — without writing the file by hand or deciding on a format each time.
Someone reviewing the harness periodically. They want to read docs/study/ offline, tally the levels, and decide whether the next improvement belongs in the command prompts, in how issues are drafted, in the codebase itself, or in the build environment. They need the files to be consistent, classified, and free of invented filler.
Capability
A command that reviews the work just done, decides whether anything about it was genuinely surprising, and — only if so — records each surprise as one row: what was found, which of four levels it sits at, and what was done about it.
The four levels, and what a recurring one implies:
Execution — the goal and the written criteria were right, the doing went wrong. Recurring: improve the command prompts, the gates, or the tooling.
Plan-spec — the criteria were wrong, ambiguous, or incomplete. Recurring: improve how issues are drafted upstream.
Codebase — the doing and the criteria were both sound; the existing code was not what anyone assumed. A hidden coupling, a landmine, debt that made the work harder than the criteria implied. Recurring: a refactor, or issues raised against the code itself.
Environment — the surprise came from outside the repository: a check that failed on a clean runner after passing locally, a tool or service that was unavailable, a dependency that moved. Recurring: harden the environment or the way the harness depends on it.
The command takes whatever evidence is available to it at the moment it runs — the live conversation when it is in the same session as the work, and otherwise the branch's commits, the diff, PR review comments including the async @claude review, and CI runs — and states which of those it actually had, so a thin record is legibly thin-because-run-late rather than thin-because-nothing-happened.
Because it can be run repeatedly against the same item, a second run adds only what is new and leaves the file otherwise untouched.
Scenario: Recording surprises after a verify
Given a piece of work whose run held surprises, When the invoker runs the command, Then each surprise is recorded as one classified row in that issue's study file, the file is committed, the working tree is left clean, and the invoker is told what was recorded and which evidence sources were available.
Scenario: A clean run records nothing
Given a piece of work that ran without surprises, When the invoker runs the command, Then no file is created, nothing is committed, and the invoker is told plainly that there was nothing to record.
Scenario: Running again after more work
Given a study file that already exists for an issue, When the invoker runs the command again after further work on it, Then only genuinely new findings are added, existing rows are left as they are, and a run with nothing new to add changes neither the file nor the working tree.
Out of scope
A read-back or tally mode is not in this issue. Reading the accumulated files and drawing conclusions from them is done offline by a human.
Mitigating evidence decay is not in this issue. Findings that exist only in a live session are lost when the command is run later from a fresh one; the accepted guidance is simply to run it in the same session as the work.
Making /build or /verify write study records themselves is not in this issue. No existing command is changed.
A Goal level — recording that the work itself was misaimed — is not in this issue. That judgement belongs at issue-drafting time, not downstream.
Acceptance criteria
Running the command with no argument on a branch named for an issue records against that issue; running it with an explicit issue number works from any feature branch.
A run that finds genuine surprises records each one as a row carrying the finding, its level, and what was done about it, in that issue's file under docs/study/.
A run that finds nothing writes no file, creates no commit, and says so explicitly — there is no "nothing notable" record.
Every recorded row is grounded in evidence the run can point to — a commit, a review comment, a failing test. A row that cannot be grounded is not recorded.
Each row is classified to exactly one of Execution, Plan-spec, Codebase, or Environment.
The final report states which evidence sources were available to the run.
Re-running against an issue that already has a file adds only new findings, leaves existing rows unchanged, and — when there is nothing new — leaves both the file and the working tree untouched.
Anything the command writes is committed, so the working tree is clean afterwards and /verify can run immediately after it without complaint.
Run on a feature branch, the record is committed to that branch so it travels with the work's pull request.
Run from main, the command refuses: it reports, writes no file and creates no commit or branch.
The command never pushes, never opens or modifies a pull request, and never merges.
The command changes no source, test, or configuration file, runs no tests, and applies no formatting.
The command never edits CLAUDE.md, project memory, or hooks. A finding that looks mechanically enforceable is flagged in the report for /capture-learnings instead.
A failure inside the command is reported and leaves the repository unchanged; it never blocks or fails anything else.
docs/study/ carries a README explaining the four levels and how to read the accumulated files back, so a later reader can interpret the classification.
Technical notes
Where it lives
A command file alongside the existing /build, /verify and /raise-pr commands.
Records under docs/study/, one file per issue, plus a README for the folder.
Integration points
Reads the current branch to infer the issue when given no argument; the existing feature/<N>-<slug> convention that /build creates carries the number.
Reads PR review comments and CI results through gh when the work has reached a pull request.
The chain script (Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270) invokes this command twice per run — once after /build and once after /verify — each time inside that stage's own session. That is an ordinary use of the re-run behaviour specified above and requires nothing extra of this command.
Constraints / assumptions
Asked what surprised it, a model will readily invent plausible surprises. Requiring evidence for every row, and stating that recording nothing is the normal outcome, is the guard against a folder that fills with confabulation and becomes worse than no record at all.
The command must leave a clean tree, because /verify refuses to start on a dirty one.
Distinguishing Codebase from Execution is a judgement call and will sometimes be got wrong. The test is where the fix belongs: a prompt or gate change is Execution, a change to the code that was already there is Codebase.
Open questions / future work
Whether the Environment level earns its place. It exists because its fix has a different owner, but NetPace has no runtime stack to fail, so it may prove near-empty — the same fault that removed Goal. Worth checking once there are enough files to tally.
Whether the record is worth keeping at all if, after some months, the accumulated files have not changed a single decision. Worth revisiting once there are enough to tally.
Confirmed decisions
Requirements
Inclusion bar and row cap: A reviewer finding that had to be actioned counts as a surprise, and there is no cap on rows per pass. (Author redirected from "bar = the surprise changed what got done, cap at 5 rows" — the sister repo's 59 files average 3.4 rows and peak at 6, so a cap solves a problem that has never occurred, and its Execution tally is only meaningful because routine review fixes were counted.)
Evidence column: Three columns only; the Finding text names its source inline where there is one, e.g. Reviewer (silent-failure-hunter): …. (Author redirected from "add a fourth Evidence column" — the sister repo's rows already self-ground this way and read as auditable.)
Re-run shape: One flat table appended to, with no per-pass sections, no dates and no identifiers; dedup is a judgement call against the existing row text. (Author redirected from "one ## section per pass headed with ISO date and stage" — a reader tallying levels never needs to know which pass wrote a row.)
Evidence-source statement: Stated in the chat report only; the file carries no evidence-sources line. (Author redirected from "record it in both the file and the report".)
Command nudges: Out of scope for this issue: no other command is altered, so AC 15 is dropped — no /verify nudge and no /build nudge. Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270 invokes /study directly, so nothing depends on a nudge existing. (Author redirected from "reword /verify's closing line and add a mention to /build".)
Un-inferable issue number: With no argument and a branch name carrying no issue number, stop and report, writing nothing and committing nothing.
Running from main: Refuse to run from main — report and write nothing. AC 10 and the study/<N> branch are dropped; /study runs on a feature branch only. (Author redirected from "commit to study/<N> and leave HEAD there" — refusing removes the branch-naming and HEAD-position questions with it.)
Technical
RED evidence: Tooling carve-out with no automated tool available — record the before/after observable behaviour in the PR body and write no bespoke markdown test. No CIR. (Author redirected from "add a CIR under docs/change-intent-records/" — docs/study/README.md carries the rationale a CIR would duplicate, and the issue body carries the rejected alternatives.)
Verdict line: End with STUDIED issue=<N> rows=<n> (rows=0 on a clean run) or FAILED reason=<short reason>, and state explicitly that the command never prompts.
File name and table shape:docs/study/<N>.md (bare number), H1 # <N> — <issue title>, and a three-column table | Finding | Level | Fix applied |. docs/study/README.md is ported from the sister repo with its three levels swapped for the four defined here. (Author redirected from "H1 # Study — #<N>: <title> and a four-column table including Evidence" — adopting the sister repo's shape verbatim is the point of the port.)
.claude/commands/capture-learnings.md — the adjacent, deliberately different reflection step: it captures corrections the invoker gave and prefers converting them into hooks or tests, where this command records what surprised the agent and only ever writes a record.
Summary
Add a
/studycommand that records what surprised a piece of work — a mid-flight redesign, a reviewer finding that had to be actioned, an acceptance criterion that turned out ambiguous — as a short, classified table indocs/study/<issue>.md. Run at any point, it complements/buildand/verifywithout being wired into either. Over time the accumulated files answer a question no single PR can: which part of the harness keeps costing us, and therefore what to fix.Motivation
/buildand/verifynow carry a piece of work from a GitHub issue to a reviewed, test-green branch. What neither leaves behind is any record of where the work went sideways on the way. A reviewer finding gets fixed and disappears into a commit; an ambiguous criterion gets interpreted and never mentioned again. Each is invisible individually, and collectively they are the only evidence of whether the harness itself needs work.A sister project running the same command set under an autonomous dispatcher has been keeping exactly this record. Fifty-nine such files accumulated there in about two weeks, and the tally across them reads 142 findings at Execution level and 60 at Plan-spec — a clear, actionable reading (the doing is lossy; the specs are mostly sound) that no individual PR could have produced. NetPace ships far more slowly, so the aggregate will take much longer to become legible here. That is an argument for starting the record cheaply now, not for waiting: files never written can never be read back.
Three design points follow from that project's experience:
Users & jobs
/buildand/verify, something during the run was unexpected, and they want it recorded before the context evaporates — without writing the file by hand or deciding on a format each time.docs/study/offline, tally the levels, and decide whether the next improvement belongs in the command prompts, in how issues are drafted, in the codebase itself, or in the build environment. They need the files to be consistent, classified, and free of invented filler.Capability
A command that reviews the work just done, decides whether anything about it was genuinely surprising, and — only if so — records each surprise as one row: what was found, which of four levels it sits at, and what was done about it.
The four levels, and what a recurring one implies:
The command takes whatever evidence is available to it at the moment it runs — the live conversation when it is in the same session as the work, and otherwise the branch's commits, the diff, PR review comments including the async
@claudereview, and CI runs — and states which of those it actually had, so a thin record is legibly thin-because-run-late rather than thin-because-nothing-happened.Because it can be run repeatedly against the same item, a second run adds only what is new and leaves the file otherwise untouched.
Scenario: Recording surprises after a verify
Given a piece of work whose run held surprises, When the invoker runs the command, Then each surprise is recorded as one classified row in that issue's study file, the file is committed, the working tree is left clean, and the invoker is told what was recorded and which evidence sources were available.
Scenario: A clean run records nothing
Given a piece of work that ran without surprises, When the invoker runs the command, Then no file is created, nothing is committed, and the invoker is told plainly that there was nothing to record.
Scenario: Running again after more work
Given a study file that already exists for an issue, When the invoker runs the command again after further work on it, Then only genuinely new findings are added, existing rows are left as they are, and a run with nothing new to add changes neither the file nor the working tree.
Out of scope
/buildor/verifywrite study records themselves is not in this issue. No existing command is changed.Acceptance criteria
docs/study/./verifycan run immediately after it without complaint.main, the command refuses: it reports, writes no file and creates no commit or branch.CLAUDE.md, project memory, or hooks. A finding that looks mechanically enforceable is flagged in the report for/capture-learningsinstead.docs/study/carries a README explaining the four levels and how to read the accumulated files back, so a later reader can interpret the classification.Technical notes
Where it lives
/build,/verifyand/raise-prcommands.docs/study/, one file per issue, plus a README for the folder.Integration points
feature/<N>-<slug>convention that/buildcreates carries the number.ghwhen the work has reached a pull request./buildand once after/verify— each time inside that stage's own session. That is an ordinary use of the re-run behaviour specified above and requires nothing extra of this command.Constraints / assumptions
/verifyrefuses to start on a dirty one.Open questions / future work
Confirmed decisions
Requirements
Reviewer (silent-failure-hunter): …. (Author redirected from "add a fourth Evidence column" — the sister repo's rows already self-ground this way and read as auditable.)##section per pass headed with ISO date and stage" — a reader tallying levels never needs to know which pass wrote a row.)/verifynudge and no/buildnudge. Add scripts/chain.sh — run the SDLC command chain end to end for one issue #270 invokes/studydirectly, so nothing depends on a nudge existing. (Author redirected from "reword/verify's closing line and add a mention to/build".)main: Refuse to run frommain— report and write nothing. AC 10 and thestudy/<N>branch are dropped;/studyruns on a feature branch only. (Author redirected from "commit tostudy/<N>and leave HEAD there" — refusing removes the branch-naming and HEAD-position questions with it.)Technical
docs/change-intent-records/" —docs/study/README.mdcarries the rationale a CIR would duplicate, and the issue body carries the rejected alternatives.)STUDIED issue=<N> rows=<n>(rows=0on a clean run) orFAILED reason=<short reason>, and state explicitly that the command never prompts.docs/study/<N>.md(bare number), H1# <N> — <issue title>, and a three-column table| Finding | Level | Fix applied |.docs/study/README.mdis ported from the sister repo with its three levels swapped for the four defined here. (Author redirected from "H1# Study — #<N>: <title>and a four-column table including Evidence" — adopting the sister repo's shape verbatim is the point of the port.)Related
ship.mduntil Split /ship into /verify and a manual /raise-pr #257 renames it)./shipinto/verifyand a manual/raise-pr. This issue is written against the post-split chain and depends on it.