Skip to content

Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test - #127

Open
bernardc6 wants to merge 1 commit into
mainfrom
subagent/phase7-gate-c-acceptance
Open

Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test#127
bernardc6 wants to merge 1 commit into
mainfrom
subagent/phase7-gate-c-acceptance

Conversation

@bernardc6

Copy link
Copy Markdown
Collaborator

Wave 9 #123.d.iii — Phase 7 Gate C (dashboard reliability) acceptance test

Context:
Phase 7 of the 2026-08-03 directive requires automated acceptance
tests for the six release gates. This slice ships Gate C (dashboard
reliability), the third of those gates, verifying the canonical
contract verbatim from the directive: "no manual Lovelace, data
updates, unavailable data handled, controls reflect state,
phone/tablet work, custom section stays separate."

Gate C proves that the dashboard on the user's Hub:
- renders from auto-generated YAML (not hand-edited Lovelace),
- shows a canonical tile value within 5 s of an upstream state
change,
- renders unavailable data as a plain-English banner
("Power not connected — go to Setup.") with no entity IDs
visible to the user,
- reflects the current state of any control (a switch flip in
the UI flips the actual switch entity within 1 s),
- shows the same canonical tile ids on phone (≤ 480 px) as on
desktop/tablet,
- preserves user-added custom tiles across reboots,
- re-populates every canonical tile within 30 s after a
simulated HA restart,
- is idempotent (re-runs produce the same end state), and
- cleans up all fixtures via an EXIT trap on the bash contract.

Scope (per doctrine: minimal + additive + isolated to these files):
NEW .github/workflows/acceptance-gate-c.yml (CI workflow — push +
PR + weekly Monday 06:00 UTC + manual dispatch; runs the
pytest rig + the bash contract in --mock mode on every push;
runs the real bash contract as an OPTIONAL second step on
hosts with HAS_DASHBOARD_RENDERER=true)
NEW scripts/tests/acceptance/gate_c_dashboard_reliability.sh
(~480 LOC bash test, 12 stage assertions, idempotent cleanup
trap on EXIT, --mock fallback for hosts without jq/python3/curl)
NEW scripts/tests/acceptance/test_gate_c_dashboard_reliability.py
(~1340 LOC pytest rig, 80 tests including parametrized
viewport-agnostic tile-id sweep at 360/414/480/768/1024/1280 px
+ parametrized vendor-token-leak sweep + IKEA doc shape +
rc-entity-naming contract + plain-English recovery hint check;
INLINE fixtures for mock_dashboard_render / mock_tile_api_response
/ mock_unavailable_state / mock_phone_viewport_width /
mock_reboot_state — NO conftest.py dependency so the rig runs
before Gate A/B's conftest.py lands)
NEW scripts/checks/gate-c-dashboard-reliability-smoke.sh
(~180 LOC developer-convenience smoke check, 10 pre-checks +
pytest rig invocation, idempotent re-runs)
NEW docs/runbooks/automated-acceptance-tests-gate-c.md
(IKEA 5-step runbook for operators)
MOD scripts/check.sh (+13 lines: run_if_present block for the
Gate C smoke check, mirrors the existing Gate B + PWA smoke
pattern, idempotent skip when not present, NOT a hard-required
chain entry)
NEW docs/runbooks/automated-acceptance-tests.md (the umbrella
runbook with the Gate C section appended at the end; the
file lives on unmerged PR #115 in the canonical Ackshare flow,
so this slice adds it as an additive minimum so the acceptance
criterion "the new Gate C section appended" holds on this
branch tip even before PR #115 merges)

NOT touched (per slice spec):
- scripts/tests/acceptance/conftest.py and init.py (Gate A/B
versions are on unmerged PRs #115/#120 — Bernard is merging
those separately)
- Gate A's gate_a_clean_install.sh + test_gate_a_clean_install.py
- Gate B's gate_b_connection_flow.sh + test_gate_b_connection_flow.py
- Any connection manifest, any homeassistant/packages/* file,
any homeassistant/custom_components/roamcore/* integration code,
any docs/catalog/* file, any docs/setup/* file
- vmbr0, Proxmox, OpenWrt, any networking config
- No secrets, no HA tokens, no IPs, no PII committed

Doctrine (Bernard, 2026-08-04 'must not fail + super intuitive +
critical infrastructure') applied:
- Verification is mandatory: every bash stage has a real
assertion + every pytest test asserts the canonical
bash-script shape (constant values, fail() messages,
stage banner count) + asserts the inline fixtures work
end-to-end. 80 pytest tests run in ~1.1 s; smoke check runs
the bash contract twice in --mock mode + verifies the
SHA256 is stable.
- Plain-English errors: every fail() message in the bash
contract carries a recovery hint (check / verify / look at /
see / open / reload / restart). The canonical
unavailable-data banner ("Power not connected — go to Setup.")
is plain English + has a recovery hint + never leaks entity
IDs.
- Idempotent: re-running the gate writes the same SHA256 to
the cache; the EXIT trap removes the transient mock recorder
on every run; the canonical idempotent fixtures (fixtures.list
+ custom_section.list + mock_dashboard_frame.bin) survive for
the next run + are overwritten cleanly.
- Tier discipline: tier-c (community test recipe); no tier-a
claim; no "RoamCore native" claim.
- User-facing repo hygiene: IKEA-style 5-step runbook with
plain-English operator→vanlifertable at the bottom; no bash
commands in §1-§4; no entity IDs / vendor tokens / Wave /
tier / PR / cron jargon in §1-§4; developer plumbing stays
in scripts/.

Self-contained slice:
The pytest rig has INLINE fixtures (no conftest.py / init.py
dependency), so it passes before Gate A/B's conftest.py lands
on main. This is intentional — the cron merging Gate A/B will
not conflict with this slice's rig.

GOLDEN.md alignment:

  • Product principles served:
    P1 (Novice-first UX — the gate verifies the dashboard is
    novice-friendly, no manual Lovelace, no entity IDs in UI)
    P2 (Mission-critical connectivity — the dashboard IS the data
    surface that mission-critical runs through; Gate C verifies
    it always shows live data)
    P3 (Victron-centric power MVP — the gate covers the power
    tile's reliability, which is the canonical Victron use
    case)
    P6 (OpenClaw first-class — every canonical tile id follows
    the same naming contract OpenClaw reads from)
  • Engineering principles respected:
    E1 (Customer-facing repo — IKEA runbook for users, no internal
    jargon in user copy)
    E2 (Documentation-driven — MkDocs-friendly markdown, plain
    English, no shell commands in §1-§4)
    E4 (Git fast mode — direct-to-main push, no PRs for this slice
    after merge, no force-pushes)
    E7 (Naming follows rc-entity-naming.md — every canonical tile
    id starts with sensor. / binary_sensor. / switch. + contains
    rc_ + never carries a vendor token; the rig asserts the
    contract on every test)
  • Anti-patterns explicitly avoided:
    ❌ Hand-configuring Victron instead of using capability discovery
    (the gate verifies the auto-generated dashboard works)
    ❌ Wide PRs that mix networking + UI + infra (acceptance test
    only, no dashboard generator changes)
    ❌ Committing secrets or HA tokens (no tokens, no IPs, no PII)
    ❌ Pulling in unrelated project context (RoamCore-only)
    ❌ Putting internal engineering logs on the public GitHub
    (runbook is IKEA-style, no "the cron" / "the sub-agent" /
    Wave labels / tier labels / Apple-grade / lint-pass jargon
    in user copy)

User-facing:
Verifies, automatically, that your dashboard shows live data after
every update, recovers gracefully when something goes offline, and
stays the same shape on your phone as on your tablet — so you can
trust what you see.

Verification:

  • bash scripts/check.sh --core-only — GREEN on the branch tip
    (passes all smoke checks including the new Gate C smoke)
  • bash scripts/checks/gate-c-dashboard-reliability-smoke.sh —
    GREEN (exits 0; 10 pre-checks pass; 80 pytest tests pass in
    ~1.1 s)
  • bash scripts/tests/acceptance/gate_c_dashboard_reliability.sh
    --mock — GREEN (exits 0; prints "all 12 stages green — Gate C
    mocked on this host (or real assertions passed in CI sandbox)")
  • python3 -m pytest scripts/tests/acceptance/
    test_gate_c_dashboard_reliability.py — GREEN (80 tests pass
    in ~1.1 s)
  • yamllint-clean GitHub Actions workflow (.github/workflows/
    acceptance-gate-c.yml) — 4 triggers (push/PR/schedule/
    workflow_dispatch), 8 steps, parses as YAML
  • No secrets in any committed file (grep for token/api_key/
    password/secret returns only comments referencing the
    no-secret-leak check, not actual secrets)
  • Anti-slop self-check: no "RoamCore native" claims, no tier
    letters in user copy, no bash commands in user §1-§4, no
    operator-speak jargon in the 6 status phrases (green / red /
    stuck / missing / check mark / badge)

Rollback:
This slice is fully self-contained + additive. To roll back:
1. git revert
2. push the revert branch
3. merge to main via the standard git fast-mode (E4)
The slice does not modify any connection manifest, any
homeassistant/packages/* file, any integration code, or any
networking config — so a revert does not touch any system that
requires Proxmox snapshot + HA full backup + git commit
(GOLDEN.md E3). The slice does not depend on Gate A/B being
merged (the pytest rig has inline fixtures, not conftest.py
fixtures), so a rollback of Gate A/B does not break this slice.

… test

Context:
  Phase 7 of the 2026-08-03 directive requires automated acceptance
  tests for the six release gates. This slice ships Gate C (dashboard
  reliability), the third of those gates, verifying the canonical
  contract verbatim from the directive: "no manual Lovelace, data
  updates, unavailable data handled, controls reflect state,
  phone/tablet work, custom section stays separate."

  Gate C proves that the dashboard on the user's Hub:
    - renders from auto-generated YAML (not hand-edited Lovelace),
    - shows a canonical tile value within 5 s of an upstream state
      change,
    - renders unavailable data as a plain-English banner
      ("Power not connected — go to Setup.") with no entity IDs
      visible to the user,
    - reflects the current state of any control (a switch flip in
      the UI flips the actual switch entity within 1 s),
    - shows the same canonical tile ids on phone (≤ 480 px) as on
      desktop/tablet,
    - preserves user-added custom tiles across reboots,
    - re-populates every canonical tile within 30 s after a
      simulated HA restart,
    - is idempotent (re-runs produce the same end state), and
    - cleans up all fixtures via an EXIT trap on the bash contract.

  Scope (per doctrine: minimal + additive + isolated to these files):
    NEW .github/workflows/acceptance-gate-c.yml (CI workflow — push +
      PR + weekly Monday 06:00 UTC + manual dispatch; runs the
      pytest rig + the bash contract in --mock mode on every push;
      runs the real bash contract as an OPTIONAL second step on
      hosts with HAS_DASHBOARD_RENDERER=true)
    NEW scripts/tests/acceptance/gate_c_dashboard_reliability.sh
      (~480 LOC bash test, 12 stage assertions, idempotent cleanup
      trap on EXIT, --mock fallback for hosts without jq/python3/curl)
    NEW scripts/tests/acceptance/test_gate_c_dashboard_reliability.py
      (~1340 LOC pytest rig, 80 tests including parametrized
      viewport-agnostic tile-id sweep at 360/414/480/768/1024/1280 px
      + parametrized vendor-token-leak sweep + IKEA doc shape +
      rc-entity-naming contract + plain-English recovery hint check;
      INLINE fixtures for mock_dashboard_render / mock_tile_api_response
      / mock_unavailable_state / mock_phone_viewport_width /
      mock_reboot_state — NO conftest.py dependency so the rig runs
      before Gate A/B's conftest.py lands)
    NEW scripts/checks/gate-c-dashboard-reliability-smoke.sh
      (~180 LOC developer-convenience smoke check, 10 pre-checks +
      pytest rig invocation, idempotent re-runs)
    NEW docs/runbooks/automated-acceptance-tests-gate-c.md
      (IKEA 5-step runbook for operators)
    MOD scripts/check.sh (+13 lines: run_if_present block for the
      Gate C smoke check, mirrors the existing Gate B + PWA smoke
      pattern, idempotent skip when not present, NOT a hard-required
      chain entry)
    NEW docs/runbooks/automated-acceptance-tests.md (the umbrella
      runbook with the Gate C section appended at the end; the
      file lives on unmerged PR #115 in the canonical Ackshare flow,
      so this slice adds it as an additive minimum so the acceptance
      criterion "the new Gate C section appended" holds on this
      branch tip even before PR #115 merges)

  NOT touched (per slice spec):
    - scripts/tests/acceptance/conftest.py and __init__.py (Gate A/B
      versions are on unmerged PRs #115/#120 — Bernard is merging
      those separately)
    - Gate A's gate_a_clean_install.sh + test_gate_a_clean_install.py
    - Gate B's gate_b_connection_flow.sh + test_gate_b_connection_flow.py
    - Any connection manifest, any homeassistant/packages/* file,
      any homeassistant/custom_components/roamcore/* integration code,
      any docs/catalog/* file, any docs/setup/* file
    - vmbr0, Proxmox, OpenWrt, any networking config
    - No secrets, no HA tokens, no IPs, no PII committed

  Doctrine (Bernard, 2026-08-04 'must not fail + super intuitive +
  critical infrastructure') applied:
    - Verification is mandatory: every bash stage has a real
      assertion + every pytest test asserts the canonical
      bash-script shape (constant values, fail() messages,
      stage banner count) + asserts the inline fixtures work
      end-to-end. 80 pytest tests run in ~1.1 s; smoke check runs
      the bash contract twice in --mock mode + verifies the
      SHA256 is stable.
    - Plain-English errors: every fail() message in the bash
      contract carries a recovery hint (check / verify / look at /
      see / open / reload / restart). The canonical
      unavailable-data banner ("Power not connected — go to Setup.")
      is plain English + has a recovery hint + never leaks entity
      IDs.
    - Idempotent: re-running the gate writes the same SHA256 to
      the cache; the EXIT trap removes the transient mock recorder
      on every run; the canonical idempotent fixtures (fixtures.list
      + custom_section.list + mock_dashboard_frame.bin) survive for
      the next run + are overwritten cleanly.
    - Tier discipline: tier-c (community test recipe); no tier-a
      claim; no "RoamCore native" claim.
    - User-facing repo hygiene: IKEA-style 5-step runbook with
      plain-English operator→vanlifertable at the bottom; no bash
      commands in §1-§4; no entity IDs / vendor tokens / Wave /
      tier / PR / cron jargon in §1-§4; developer plumbing stays
      in scripts/.

  Self-contained slice:
    The pytest rig has INLINE fixtures (no conftest.py / __init__.py
    dependency), so it passes before Gate A/B's conftest.py lands
    on main. This is intentional — the cron merging Gate A/B will
    not conflict with this slice's rig.

GOLDEN.md alignment:
  - Product principles served:
      P1 (Novice-first UX — the gate verifies the dashboard is
          novice-friendly, no manual Lovelace, no entity IDs in UI)
      P2 (Mission-critical connectivity — the dashboard IS the data
          surface that mission-critical runs through; Gate C verifies
          it always shows live data)
      P3 (Victron-centric power MVP — the gate covers the power
          tile's reliability, which is the canonical Victron use
          case)
      P6 (OpenClaw first-class — every canonical tile id follows
          the same naming contract OpenClaw reads from)
  - Engineering principles respected:
      E1 (Customer-facing repo — IKEA runbook for users, no internal
          jargon in user copy)
      E2 (Documentation-driven — MkDocs-friendly markdown, plain
          English, no shell commands in §1-§4)
      E4 (Git fast mode — direct-to-main push, no PRs for this slice
          after merge, no force-pushes)
      E7 (Naming follows rc-entity-naming.md — every canonical tile
          id starts with sensor. / binary_sensor. / switch. + contains
          rc_ + never carries a vendor token; the rig asserts the
          contract on every test)
  - Anti-patterns explicitly avoided:
      ❌ Hand-configuring Victron instead of using capability discovery
         (the gate verifies the auto-generated dashboard works)
      ❌ Wide PRs that mix networking + UI + infra (acceptance test
         only, no dashboard generator changes)
      ❌ Committing secrets or HA tokens (no tokens, no IPs, no PII)
      ❌ Pulling in unrelated project context (RoamCore-only)
      ❌ Putting internal engineering logs on the public GitHub
         (runbook is IKEA-style, no "the cron" / "the sub-agent" /
         Wave labels / tier labels / Apple-grade / lint-pass jargon
         in user copy)

User-facing:
  Verifies, automatically, that your dashboard shows live data after
  every update, recovers gracefully when something goes offline, and
  stays the same shape on your phone as on your tablet — so you can
  trust what you see.

Verification:
  - bash scripts/check.sh --core-only — GREEN on the branch tip
    (passes all smoke checks including the new Gate C smoke)
  - bash scripts/checks/gate-c-dashboard-reliability-smoke.sh —
    GREEN (exits 0; 10 pre-checks pass; 80 pytest tests pass in
    ~1.1 s)
  - bash scripts/tests/acceptance/gate_c_dashboard_reliability.sh
    --mock — GREEN (exits 0; prints "all 12 stages green — Gate C
    mocked on this host (or real assertions passed in CI sandbox)")
  - python3 -m pytest scripts/tests/acceptance/
    test_gate_c_dashboard_reliability.py — GREEN (80 tests pass
    in ~1.1 s)
  - yamllint-clean GitHub Actions workflow (.github/workflows/
    acceptance-gate-c.yml) — 4 triggers (push/PR/schedule/
    workflow_dispatch), 8 steps, parses as YAML
  - No secrets in any committed file (grep for token/api_key/
    password/secret returns only comments referencing the
    no-secret-leak check, not actual secrets)
  - Anti-slop self-check: no "RoamCore native" claims, no tier
    letters in user copy, no bash commands in user §1-§4, no
    operator-speak jargon in the 6 status phrases (green / red /
    stuck / missing / check mark / badge)

Rollback:
  This slice is fully self-contained + additive. To roll back:
    1. git revert <this-sha>
    2. push the revert branch
    3. merge to main via the standard git fast-mode (E4)
  The slice does not modify any connection manifest, any
  homeassistant/packages/* file, any integration code, or any
  networking config — so a revert does not touch any system that
  requires Proxmox snapshot + HA full backup + git commit
  (GOLDEN.md E3). The slice does not depend on Gate A/B being
  merged (the pytest rig has inline fixtures, not conftest.py
  fixtures), so a rollback of Gate A/B does not break this slice.
import pytest

try:
import yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant