Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test - #127
Open
bernardc6 wants to merge 1 commit into
Open
Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test#127bernardc6 wants to merge 1 commit into
bernardc6 wants to merge 1 commit into
Conversation
… test
Context:
Phase 7 of the 2026-08-03 directive requires automated acceptance
tests for the six release gates. This slice ships Gate C (dashboard
reliability), the third of those gates, verifying the canonical
contract verbatim from the directive: "no manual Lovelace, data
updates, unavailable data handled, controls reflect state,
phone/tablet work, custom section stays separate."
Gate C proves that the dashboard on the user's Hub:
- renders from auto-generated YAML (not hand-edited Lovelace),
- shows a canonical tile value within 5 s of an upstream state
change,
- renders unavailable data as a plain-English banner
("Power not connected — go to Setup.") with no entity IDs
visible to the user,
- reflects the current state of any control (a switch flip in
the UI flips the actual switch entity within 1 s),
- shows the same canonical tile ids on phone (≤ 480 px) as on
desktop/tablet,
- preserves user-added custom tiles across reboots,
- re-populates every canonical tile within 30 s after a
simulated HA restart,
- is idempotent (re-runs produce the same end state), and
- cleans up all fixtures via an EXIT trap on the bash contract.
Scope (per doctrine: minimal + additive + isolated to these files):
NEW .github/workflows/acceptance-gate-c.yml (CI workflow — push +
PR + weekly Monday 06:00 UTC + manual dispatch; runs the
pytest rig + the bash contract in --mock mode on every push;
runs the real bash contract as an OPTIONAL second step on
hosts with HAS_DASHBOARD_RENDERER=true)
NEW scripts/tests/acceptance/gate_c_dashboard_reliability.sh
(~480 LOC bash test, 12 stage assertions, idempotent cleanup
trap on EXIT, --mock fallback for hosts without jq/python3/curl)
NEW scripts/tests/acceptance/test_gate_c_dashboard_reliability.py
(~1340 LOC pytest rig, 80 tests including parametrized
viewport-agnostic tile-id sweep at 360/414/480/768/1024/1280 px
+ parametrized vendor-token-leak sweep + IKEA doc shape +
rc-entity-naming contract + plain-English recovery hint check;
INLINE fixtures for mock_dashboard_render / mock_tile_api_response
/ mock_unavailable_state / mock_phone_viewport_width /
mock_reboot_state — NO conftest.py dependency so the rig runs
before Gate A/B's conftest.py lands)
NEW scripts/checks/gate-c-dashboard-reliability-smoke.sh
(~180 LOC developer-convenience smoke check, 10 pre-checks +
pytest rig invocation, idempotent re-runs)
NEW docs/runbooks/automated-acceptance-tests-gate-c.md
(IKEA 5-step runbook for operators)
MOD scripts/check.sh (+13 lines: run_if_present block for the
Gate C smoke check, mirrors the existing Gate B + PWA smoke
pattern, idempotent skip when not present, NOT a hard-required
chain entry)
NEW docs/runbooks/automated-acceptance-tests.md (the umbrella
runbook with the Gate C section appended at the end; the
file lives on unmerged PR #115 in the canonical Ackshare flow,
so this slice adds it as an additive minimum so the acceptance
criterion "the new Gate C section appended" holds on this
branch tip even before PR #115 merges)
NOT touched (per slice spec):
- scripts/tests/acceptance/conftest.py and __init__.py (Gate A/B
versions are on unmerged PRs #115/#120 — Bernard is merging
those separately)
- Gate A's gate_a_clean_install.sh + test_gate_a_clean_install.py
- Gate B's gate_b_connection_flow.sh + test_gate_b_connection_flow.py
- Any connection manifest, any homeassistant/packages/* file,
any homeassistant/custom_components/roamcore/* integration code,
any docs/catalog/* file, any docs/setup/* file
- vmbr0, Proxmox, OpenWrt, any networking config
- No secrets, no HA tokens, no IPs, no PII committed
Doctrine (Bernard, 2026-08-04 'must not fail + super intuitive +
critical infrastructure') applied:
- Verification is mandatory: every bash stage has a real
assertion + every pytest test asserts the canonical
bash-script shape (constant values, fail() messages,
stage banner count) + asserts the inline fixtures work
end-to-end. 80 pytest tests run in ~1.1 s; smoke check runs
the bash contract twice in --mock mode + verifies the
SHA256 is stable.
- Plain-English errors: every fail() message in the bash
contract carries a recovery hint (check / verify / look at /
see / open / reload / restart). The canonical
unavailable-data banner ("Power not connected — go to Setup.")
is plain English + has a recovery hint + never leaks entity
IDs.
- Idempotent: re-running the gate writes the same SHA256 to
the cache; the EXIT trap removes the transient mock recorder
on every run; the canonical idempotent fixtures (fixtures.list
+ custom_section.list + mock_dashboard_frame.bin) survive for
the next run + are overwritten cleanly.
- Tier discipline: tier-c (community test recipe); no tier-a
claim; no "RoamCore native" claim.
- User-facing repo hygiene: IKEA-style 5-step runbook with
plain-English operator→vanlifertable at the bottom; no bash
commands in §1-§4; no entity IDs / vendor tokens / Wave /
tier / PR / cron jargon in §1-§4; developer plumbing stays
in scripts/.
Self-contained slice:
The pytest rig has INLINE fixtures (no conftest.py / __init__.py
dependency), so it passes before Gate A/B's conftest.py lands
on main. This is intentional — the cron merging Gate A/B will
not conflict with this slice's rig.
GOLDEN.md alignment:
- Product principles served:
P1 (Novice-first UX — the gate verifies the dashboard is
novice-friendly, no manual Lovelace, no entity IDs in UI)
P2 (Mission-critical connectivity — the dashboard IS the data
surface that mission-critical runs through; Gate C verifies
it always shows live data)
P3 (Victron-centric power MVP — the gate covers the power
tile's reliability, which is the canonical Victron use
case)
P6 (OpenClaw first-class — every canonical tile id follows
the same naming contract OpenClaw reads from)
- Engineering principles respected:
E1 (Customer-facing repo — IKEA runbook for users, no internal
jargon in user copy)
E2 (Documentation-driven — MkDocs-friendly markdown, plain
English, no shell commands in §1-§4)
E4 (Git fast mode — direct-to-main push, no PRs for this slice
after merge, no force-pushes)
E7 (Naming follows rc-entity-naming.md — every canonical tile
id starts with sensor. / binary_sensor. / switch. + contains
rc_ + never carries a vendor token; the rig asserts the
contract on every test)
- Anti-patterns explicitly avoided:
❌ Hand-configuring Victron instead of using capability discovery
(the gate verifies the auto-generated dashboard works)
❌ Wide PRs that mix networking + UI + infra (acceptance test
only, no dashboard generator changes)
❌ Committing secrets or HA tokens (no tokens, no IPs, no PII)
❌ Pulling in unrelated project context (RoamCore-only)
❌ Putting internal engineering logs on the public GitHub
(runbook is IKEA-style, no "the cron" / "the sub-agent" /
Wave labels / tier labels / Apple-grade / lint-pass jargon
in user copy)
User-facing:
Verifies, automatically, that your dashboard shows live data after
every update, recovers gracefully when something goes offline, and
stays the same shape on your phone as on your tablet — so you can
trust what you see.
Verification:
- bash scripts/check.sh --core-only — GREEN on the branch tip
(passes all smoke checks including the new Gate C smoke)
- bash scripts/checks/gate-c-dashboard-reliability-smoke.sh —
GREEN (exits 0; 10 pre-checks pass; 80 pytest tests pass in
~1.1 s)
- bash scripts/tests/acceptance/gate_c_dashboard_reliability.sh
--mock — GREEN (exits 0; prints "all 12 stages green — Gate C
mocked on this host (or real assertions passed in CI sandbox)")
- python3 -m pytest scripts/tests/acceptance/
test_gate_c_dashboard_reliability.py — GREEN (80 tests pass
in ~1.1 s)
- yamllint-clean GitHub Actions workflow (.github/workflows/
acceptance-gate-c.yml) — 4 triggers (push/PR/schedule/
workflow_dispatch), 8 steps, parses as YAML
- No secrets in any committed file (grep for token/api_key/
password/secret returns only comments referencing the
no-secret-leak check, not actual secrets)
- Anti-slop self-check: no "RoamCore native" claims, no tier
letters in user copy, no bash commands in user §1-§4, no
operator-speak jargon in the 6 status phrases (green / red /
stuck / missing / check mark / badge)
Rollback:
This slice is fully self-contained + additive. To roll back:
1. git revert <this-sha>
2. push the revert branch
3. merge to main via the standard git fast-mode (E4)
The slice does not modify any connection manifest, any
homeassistant/packages/* file, any integration code, or any
networking config — so a revert does not touch any system that
requires Proxmox snapshot + HA full backup + git commit
(GOLDEN.md E3). The slice does not depend on Gate A/B being
merged (the pytest rig has inline fixtures, not conftest.py
fixtures), so a rollback of Gate A/B does not break this slice.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test
Context:
Phase 7 of the 2026-08-03 directive requires automated acceptance
tests for the six release gates. This slice ships Gate C (dashboard
reliability), the third of those gates, verifying the canonical
contract verbatim from the directive: "no manual Lovelace, data
updates, unavailable data handled, controls reflect state,
phone/tablet work, custom section stays separate."
Gate C proves that the dashboard on the user's Hub:
- renders from auto-generated YAML (not hand-edited Lovelace),
- shows a canonical tile value within 5 s of an upstream state
change,
- renders unavailable data as a plain-English banner
("Power not connected — go to Setup.") with no entity IDs
visible to the user,
- reflects the current state of any control (a switch flip in
the UI flips the actual switch entity within 1 s),
- shows the same canonical tile ids on phone (≤ 480 px) as on
desktop/tablet,
- preserves user-added custom tiles across reboots,
- re-populates every canonical tile within 30 s after a
simulated HA restart,
- is idempotent (re-runs produce the same end state), and
- cleans up all fixtures via an EXIT trap on the bash contract.
Scope (per doctrine: minimal + additive + isolated to these files):
NEW .github/workflows/acceptance-gate-c.yml (CI workflow — push +
PR + weekly Monday 06:00 UTC + manual dispatch; runs the
pytest rig + the bash contract in --mock mode on every push;
runs the real bash contract as an OPTIONAL second step on
hosts with HAS_DASHBOARD_RENDERER=true)
NEW scripts/tests/acceptance/gate_c_dashboard_reliability.sh
(~480 LOC bash test, 12 stage assertions, idempotent cleanup
trap on EXIT, --mock fallback for hosts without jq/python3/curl)
NEW scripts/tests/acceptance/test_gate_c_dashboard_reliability.py
(~1340 LOC pytest rig, 80 tests including parametrized
viewport-agnostic tile-id sweep at 360/414/480/768/1024/1280 px
+ parametrized vendor-token-leak sweep + IKEA doc shape +
rc-entity-naming contract + plain-English recovery hint check;
INLINE fixtures for mock_dashboard_render / mock_tile_api_response
/ mock_unavailable_state / mock_phone_viewport_width /
mock_reboot_state — NO conftest.py dependency so the rig runs
before Gate A/B's conftest.py lands)
NEW scripts/checks/gate-c-dashboard-reliability-smoke.sh
(~180 LOC developer-convenience smoke check, 10 pre-checks +
pytest rig invocation, idempotent re-runs)
NEW docs/runbooks/automated-acceptance-tests-gate-c.md
(IKEA 5-step runbook for operators)
MOD scripts/check.sh (+13 lines: run_if_present block for the
Gate C smoke check, mirrors the existing Gate B + PWA smoke
pattern, idempotent skip when not present, NOT a hard-required
chain entry)
NEW docs/runbooks/automated-acceptance-tests.md (the umbrella
runbook with the Gate C section appended at the end; the
file lives on unmerged PR #115 in the canonical Ackshare flow,
so this slice adds it as an additive minimum so the acceptance
criterion "the new Gate C section appended" holds on this
branch tip even before PR #115 merges)
NOT touched (per slice spec):
- scripts/tests/acceptance/conftest.py and init.py (Gate A/B
versions are on unmerged PRs #115/#120 — Bernard is merging
those separately)
- Gate A's gate_a_clean_install.sh + test_gate_a_clean_install.py
- Gate B's gate_b_connection_flow.sh + test_gate_b_connection_flow.py
- Any connection manifest, any homeassistant/packages/* file,
any homeassistant/custom_components/roamcore/* integration code,
any docs/catalog/* file, any docs/setup/* file
- vmbr0, Proxmox, OpenWrt, any networking config
- No secrets, no HA tokens, no IPs, no PII committed
Doctrine (Bernard, 2026-08-04 'must not fail + super intuitive +
critical infrastructure') applied:
- Verification is mandatory: every bash stage has a real
assertion + every pytest test asserts the canonical
bash-script shape (constant values, fail() messages,
stage banner count) + asserts the inline fixtures work
end-to-end. 80 pytest tests run in ~1.1 s; smoke check runs
the bash contract twice in --mock mode + verifies the
SHA256 is stable.
- Plain-English errors: every fail() message in the bash
contract carries a recovery hint (check / verify / look at /
see / open / reload / restart). The canonical
unavailable-data banner ("Power not connected — go to Setup.")
is plain English + has a recovery hint + never leaks entity
IDs.
- Idempotent: re-running the gate writes the same SHA256 to
the cache; the EXIT trap removes the transient mock recorder
on every run; the canonical idempotent fixtures (fixtures.list
+ custom_section.list + mock_dashboard_frame.bin) survive for
the next run + are overwritten cleanly.
- Tier discipline: tier-c (community test recipe); no tier-a
claim; no "RoamCore native" claim.
- User-facing repo hygiene: IKEA-style 5-step runbook with
plain-English operator→vanlifertable at the bottom; no bash
commands in §1-§4; no entity IDs / vendor tokens / Wave /
tier / PR / cron jargon in §1-§4; developer plumbing stays
in scripts/.
Self-contained slice:
The pytest rig has INLINE fixtures (no conftest.py / init.py
dependency), so it passes before Gate A/B's conftest.py lands
on main. This is intentional — the cron merging Gate A/B will
not conflict with this slice's rig.
GOLDEN.md alignment:
P1 (Novice-first UX — the gate verifies the dashboard is
novice-friendly, no manual Lovelace, no entity IDs in UI)
P2 (Mission-critical connectivity — the dashboard IS the data
surface that mission-critical runs through; Gate C verifies
it always shows live data)
P3 (Victron-centric power MVP — the gate covers the power
tile's reliability, which is the canonical Victron use
case)
P6 (OpenClaw first-class — every canonical tile id follows
the same naming contract OpenClaw reads from)
E1 (Customer-facing repo — IKEA runbook for users, no internal
jargon in user copy)
E2 (Documentation-driven — MkDocs-friendly markdown, plain
English, no shell commands in §1-§4)
E4 (Git fast mode — direct-to-main push, no PRs for this slice
after merge, no force-pushes)
E7 (Naming follows rc-entity-naming.md — every canonical tile
id starts with sensor. / binary_sensor. / switch. + contains
rc_ + never carries a vendor token; the rig asserts the
contract on every test)
❌ Hand-configuring Victron instead of using capability discovery
(the gate verifies the auto-generated dashboard works)
❌ Wide PRs that mix networking + UI + infra (acceptance test
only, no dashboard generator changes)
❌ Committing secrets or HA tokens (no tokens, no IPs, no PII)
❌ Pulling in unrelated project context (RoamCore-only)
❌ Putting internal engineering logs on the public GitHub
(runbook is IKEA-style, no "the cron" / "the sub-agent" /
Wave labels / tier labels / Apple-grade / lint-pass jargon
in user copy)
User-facing:
Verifies, automatically, that your dashboard shows live data after
every update, recovers gracefully when something goes offline, and
stays the same shape on your phone as on your tablet — so you can
trust what you see.
Verification:
(passes all smoke checks including the new Gate C smoke)
GREEN (exits 0; 10 pre-checks pass; 80 pytest tests pass in
~1.1 s)
--mock — GREEN (exits 0; prints "all 12 stages green — Gate C
mocked on this host (or real assertions passed in CI sandbox)")
test_gate_c_dashboard_reliability.py — GREEN (80 tests pass
in ~1.1 s)
acceptance-gate-c.yml) — 4 triggers (push/PR/schedule/
workflow_dispatch), 8 steps, parses as YAML
password/secret returns only comments referencing the
no-secret-leak check, not actual secrets)
letters in user copy, no bash commands in user §1-§4, no
operator-speak jargon in the 6 status phrases (green / red /
stuck / missing / check mark / badge)
Rollback:
This slice is fully self-contained + additive. To roll back:
1. git revert
2. push the revert branch
3. merge to main via the standard git fast-mode (E4)
The slice does not modify any connection manifest, any
homeassistant/packages/* file, any integration code, or any
networking config — so a revert does not touch any system that
requires Proxmox snapshot + HA full backup + git commit
(GOLDEN.md E3). The slice does not depend on Gate A/B being
merged (the pytest rig has inline fixtures, not conftest.py
fixtures), so a rollback of Gate A/B does not break this slice.