Wave 9 #123.d.i — Phase 7 Acceptance Tests foundation + Gate A (clean install) - #115
Open
bernardc6 wants to merge 1 commit into
Open
Wave 9 #123.d.i — Phase 7 Acceptance Tests foundation + Gate A (clean install)#115bernardc6 wants to merge 1 commit into
bernardc6 wants to merge 1 commit into
Conversation
… install)
Context:
Phase 7 of the 2026-08-03 directive requires automated acceptance
tests for the six release gates (A: clean install, B: connection
flow, C: dashboard reliability, D: agent integration, E: remote
access, F: recovery). This slice delivers the foundation + Gate A
only, following the 'one slice per gate' pattern so future slices
(B-F) can land incrementally.
Scope (per doctrine: minimal + additive + isolated to these files):
NEW .github/workflows/acceptance-gate-a.yml
NEW scripts/tests/acceptance/gate_a_clean_install.sh (real test)
NEW scripts/tests/acceptance/test_gate_a_clean_install.py (pytest)
NEW scripts/tests/acceptance/conftest.py (fixtures)
NEW scripts/tests/acceptance/__init__.py (package marker)
NEW scripts/tests/acceptance/README.md (operator runbook)
NEW docs/runbooks/automated-acceptance-tests.md (user IKEA doc)
NOT touched: scripts/check.sh (acceptance test runs via GitHub
Actions + standalone pytest, not via check.sh per spec).
Changes:
- .github/workflows/acceptance-gate-a.yml: GitHub Actions workflow
that runs the pytest rig on every push to main + every PR +
manual dispatch. Runs the real bash test as an optional second
step on hosts with HAS_HAOS_SANDBOX=true (self-hosted runners).
30-min timeout; valid YAML (yamllint-clean); no secrets.
- scripts/tests/acceptance/gate_a_clean_install.sh: the REAL bash
acceptance test for Gate A. Six plain-English steps:
Step 1 — Download HAOS 14.1 generic-x86-64 (or use cached)
Step 2 — Boot HAOS in qemu/kvm (with -daemonize + -pidfile)
Step 3 — Wait for Hub to respond on :8123 (120s timeout)
Step 4 — Verify the RoamCore integration is detected
Step 5 — Verify the setup wizard URL is reachable
Step 6 — Tear down (kill qemu via cleanup EXIT trap)
Each step has a plain-English failure message (no errno jargon);
idempotent (reuses cached HAOS image when SHA matches); script-
only delivery on hosts without qemu (exits 0 with a plain-English
skip message). Pinned to the same SHA as
scripts/build/hub-golden-image.manifest.yml.
- scripts/tests/acceptance/test_gate_a_clean_install.py: the pytest
rig. 10 tests covering all six steps + the idempotency contract +
the script-only-delivery path. All tests are repo-local (no
network, no root, no /tmp leak). Each assertion quotes the
contract element it guards, with a docstring explaining the
rationale + the recovery path. End-to-end test invokes the bash
script via subprocess.run + asserts the plain-English skip
message.
- scripts/tests/acceptance/conftest.py: pytest fixtures (the bash
script path, a MagicMock for subprocess.run, a canned HAOS
response, a pinned SHA, a canned onboarding HTML). Repo-local
only; resets before every test.
- scripts/tests/acceptance/__init__.py: package marker (the
directory needs to be a Python package for pytest collection).
- scripts/tests/acceptance/README.md: operator-facing runbook.
Five-step IKEA shape: what this is / what you see / what you do
/ what to do if it goes wrong / useful links. Includes bash
commands (operator-facing exception per the user-vs-operator
rule — this file lives in scripts/, not docs/).
- docs/runbooks/automated-acceptance-tests.md: USER-FACING IKEA
runbook (the only file in this slice that lives in docs/).
Opens with one plain-English sentence ('Every new release of
RoamCore is tested automatically before it goes out...'). Five-
step IKEA shape. Operator→vanlifer translation table honored
(acceptance test → automatic install test, CI → automatic test
runner, gate → checkpoint, sandbox → test environment). NO file
paths, function names, PR numbers, bash commands in §1-§4, tier
letters, or internal jargon. Closes with a glossary paragraph
that translates the four key terms in plain English.
Verification:
- bash scripts/check.sh --core-only: GREEN (exit 0)
(this slice does NOT modify check.sh; the existing chain still
passes with the new files added)
- pytest scripts/tests/acceptance/test_gate_a_clean_install.py -v:
10/10 PASS on this host (mocked subprocess; no qemu needed)
- bash scripts/tests/acceptance/gate_a_clean_install.sh:
exits 0 on this host (script-only delivery; 'QEMU not available
— Gate A runs in CI sandbox only' message; plain-English skip)
- shellcheck scripts/tests/acceptance/gate_a_clean_install.sh:
exit 0 (clean; SC2329 disable for trap-based cleanup is the only
shellcheck directive needed)
- yamllint .github/workflows/acceptance-gate-a.yml:
exit 0 (only the GitHub-Actions-typical document-start + truthy
'on:' warnings, which are standard for Actions workflows)
- bash scripts/tests/acceptance/gate_a_clean_install.sh (re-run):
exits 0 — idempotency contract holds
- No secrets in any file (test fixtures use mocked data + canned
responses only)
- User-facing doc self-check:
* Opens with one plain-English sentence: PASS
* No file paths / function names / PR numbers / Wave labels /
tier letters / 'RoamCore native' claims: PASS
* Operator→vanlifer translation table honored: PASS
* No bash commands in §1-§4: PASS
Rollback:
git revert <sha> -- .github/workflows/acceptance-gate-a.yml
scripts/tests/acceptance/
docs/runbooks/automated-acceptance-tests.md
OR
git reset --hard <previous-sha> # if branch is unpushed
The slice is fully additive — rollback removes only the new files
and leaves the existing repo unchanged. No migrations to undo; no
external state to clean up (the bash test caches the HAOS image at
${ROAMCORE_GATE_A_CACHE:-.cache/gate-a}, which gitignores the
.cache/ directory already).
User-facing:
RoamCore's automated tests prove that every new release still
installs cleanly on a fresh Hub, so I never receive an update
that breaks my install.
GOLDEN.md alignment:
Product principles served:
P1 (Novice-first UX): Gate A 'clean install' proves the novice
install path is always green; users never see a broken install.
Quote: 'A van owner should be able to glance at a single
dashboard and know whether everything is fine' — Gate A is
the prerequisite: the dashboard cannot load until the clean
install completes successfully.
P2 (Mission-critical connectivity): Gate A acceptance test gates
every release against the 'fresh Hub boots, services start,
no terminal' contract. Quote: 'LTE primary, Starlink failover,
Tailscale-like remote access. Network failures must not lose
data or lock the user out.' — A broken clean install is the
most locking-out failure mode possible; Gate A is the guard.
Engineering principles respected:
E1 (Customer-facing repo): The user-facing IKEA doc lands at
docs/runbooks/automated-acceptance-tests.md (correct location,
per the user-tree rule). The developer plumbing lives in
scripts/tests/acceptance/ + .github/workflows/ (internal
locations). No SUPERSEDED banners in the user tree; no
Cron-handoff folders in the user tree.
E2 (Documentation-driven): Each gate ships with a user-facing
IKEA doc + an operator-facing runbook (the scripts/tests/
acceptance/README.md is the operator-facing side; docs/
runbooks/automated-acceptance-tests.md is the user-facing
side).
E3 (Backup + rollback discipline): Gate A runs in a CI sandbox
only — never touches production. The bash test has a cleanup
EXIT trap that kills qemu on every exit (success + failure)
so re-runs do not leak processes. Idempotent (re-runs reuse
the cached HAOS image when SHA matches).
E4 (Git fast mode): The GitHub Actions workflow runs on every
push to main + every PR — Gate A is part of the CI gate chain.
The slice lands on a subagent/* branch (one gate per slice)
per the direct-to-main workflow.
Anti-patterns explicitly avoided:
❌ Hand-configuring Victron (purely acceptance test infra — no
Victron integration touched)
❌ Touching vmbr0 (no networking config touched; the test runs
in a CI sandbox only)
❌ Committing secrets (test fixtures use mocked data + canned
responses only; no real tokens)
❌ Wide PRs (focused acceptance-tests slice: 7 new files, zero
modifications to existing files)
❌ Internal engineering logs on public GitHub (IKEA runbook;
no SUPERSEDED banners in user tree)
❌ Adding 'advanced' features before the novice path is solid —
Gate A IS the novice-path guarantee (it proves a fresh Hub
boots cleanly without requiring the operator to touch a
terminal)
Tier discipline (tier-c): the test infrastructure is upstream pytest
+ GitHub Actions (community test recipe), not a RoamCore-native test
engine. The bash script + the pytest rig + the GitHub Actions
workflow are all standard, off-the-shelf test plumbing.
This was referenced Aug 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 9 #123.d.i — Phase 7 Acceptance Tests foundation + Gate A (clean install)
Phase 7 of the 2026-08-03 directive. This slice delivers the
foundation for the six release gates + Gate A (clean install)
end-to-end. Future slices will add Gates B-F following the same
shape.
Scope (tight — these files ONLY)
.github/workflows/acceptance-gate-a.yml— GitHub Actions workflow for Gate Ascripts/tests/acceptance/gate_a_clean_install.sh— real bash acceptance test (307 LOC)scripts/tests/acceptance/test_gate_a_clean_install.py— pytest rig with mocked subprocess (472 LOC, 10 tests)scripts/tests/acceptance/conftest.py— pytest fixtures (173 LOC)scripts/tests/acceptance/__init__.py— package marker (16 LOC)scripts/tests/acceptance/README.md— operator-facing runbook (IKEA 5-step)docs/runbooks/automated-acceptance-tests.md— user-facing runbook (IKEA 5-step)NOT touched:
scripts/check.sh(acceptance test runs via GitHub Actions + standalone pytest, not via check.sh per spec). The existingbash scripts/check.sh --core-onlychain still passes GREEN.Acceptance checklist
on:truthy + missing document-start warnings, which are standard)Tier-discipline
Per GOLDEN.md: tier-c — community test recipe over the test rig. The
test infrastructure is upstream pytest + GitHub Actions (the
off-the-shelf test plumbing), not a RoamCore-native test engine.
Idempotency proof
Both runs exit 0 with the same plain-English message.
Rollback plan
The slice is fully additive — rollback removes only the new files
and leaves the existing repo unchanged. No migrations to undo; no
external state to clean up (the bash test caches the HAOS image at
${ROAMCORE_GATE_A_CACHE:-.cache/gate-a}, which is alreadygitignored).
GOLDEN.md alignment (full quote in commit body)
Product principles served:
Engineering principles respected:
docs/runbooks/automated-acceptance-tests.md(correct location, per the user-tree rule). Developer plumbing lives inscripts/tests/acceptance/+.github/workflows/(internal locations).Anti-patterns explicitly avoided:
Reviewer
@bernardc6
Reference
/home/bernard/clawd/RoamCore/GOLDEN.md(constitution)/home/bernard/.openclaw/workspace/memory/roamcore/2026-08-03-directive.md(Phase 7 / Gate F section)scripts/build/hub-golden-image.sh(the script-only-delivery pattern)connections/openclaw-api/tests/(the pytest rig pattern)