Skip to content

Wave 9 #119d — Phase 2 canonical vehicle model verification framework - #123

Open
bernardc6 wants to merge 3 commits into
mainfrom
subagent/phase2-verification-framework
Open

Wave 9 #119d — Phase 2 canonical vehicle model verification framework#123
bernardc6 wants to merge 3 commits into
mainfrom
subagent/phase2-verification-framework

Conversation

@bernardc6

Copy link
Copy Markdown
Collaborator

Wave 9 #119d — Phase 2 canonical vehicle model verification framework

This slice ships the post-install verification framework for the Phase 2 canonical vehicle model. Per Directive Rule 4 ("Don't mark successful if automated action can't be verified"), successful install != working connection. This framework turns "the tile is showing" into "the tile is showing REAL DATA that we just verified".

Sibling slice to:

  • #119a — schema primitive (landed at d5138ed)
  • #119b — mapping layer (landed at 1a98605)
  • #119c — dashboard generator (in flight)
  • #119d — this slice — verification framework

What ships

A pure-stdlib + json framework (homeassistant/custom_components/roamcore/verification.py) with all 5 verification checks:

  1. value_in_range — most recent sample within expected_range (derived from canonical capability's unit + device_class: battery % → 0..100, voltage → 0..60, current → -500..500, power → -5000..5000, temperature → -50..80, latitude → -90..90, longitude → -180..180).
  2. recent_update — at least one sample in the last 60s (DEFAULT_FRESHNESS_SECONDS).
  3. data_plausible — no impossible sample-to-sample jumps (any delta > 50% of expected range in < 60s fails).
  4. restart_resilience — restart_count >= 0 AND if last_disconnect_at is set, samples exist AFTER it.
  5. vendor_match — entity_id's prefix matches one of the canonical example_sources families (substring match, not regex).

Public surface: VerificationTarget + VerificationResult dataclasses (frozen), verify_connection() top-level, verify_capability_map() batch helper, assert_rc_prefix() + assert_no_vendor_tokens() guards. All reason + recovery_hint strings are one-sentence plain English (operator→vanlifer translation table enforced).

Verification

  • bash scripts/check.sh --core-only GREEN (exit 0)
  • python3 -m pytest homeassistant/custom_components/roamcore/tests/test_verification.py -v52 tests PASS in 0.07s (≥30 required)
  • bash scripts/checks/verification-framework-smoke.sh → 8+ bash assertions PASS
  • ✅ Pure stdlib + json (no HA imports — verified by AST scan in the bash smoke)
  • ✅ rc-entity-naming honoured for all 15 shipped canonical ids + 4 runtime test rig ids
  • ✅ All reason + recovery_hint strings pass the operator→vanlifer translation table
  • ✅ User-facing IKEA doc at docs/reference/rc-verification.md (5 sections, plain English, translation table at §5)

Files changed

  • NEW homeassistant/custom_components/roamcore/verification.py (~470 LOC)
  • NEW homeassistant/custom_components/roamcore/tests/test_verification.py (~850 LOC, 52 pytest tests)
  • NEW scripts/checks/verification-framework-smoke.sh (~250 LOC, 8+ bash assertions)
  • NEW docs/reference/rc-verification.md (~110 LOC, IKEA 5-step)
  • MODIFIED scripts/check.sh (+1 run_if_present line)
  • MODIFIED mkdocs.yml (+1 nav entry)

6 files changed, 2136 insertions, 0 deletions.

Rollback

Revert this commit; remove the run_if_present line in scripts/check.sh; remove the mkdocs.yml nav entry.

User-facing one-liner

Checks that each device is actually sending fresh, sensible data — not just that the card is on the screen — and tells you honestly in plain English if anything looks wrong, with one sentence about what to do next.

openclaw-subagent and others added 3 commits August 10, 2026 04:47
…el → phone → tunnel → HA)

User-facing: Lets me confirm with one tap that I can reach my van from
anywhere — and tells me honestly if the tunnel isn't working both ways.

Context

  This slice lands the Gate E 'tunnel works both ways' sub-slice of
  the Phase 6 Tailscale wizard (Wave 9 #122.d.iii). The previous
  slice #122.a (commit 5f94405, PR #108) wired Path A (Tailscale)
  end-to-end + shipped Path B/C/D as stubs; this slice adds a
  one-tap self-test that proves the tunnel really works in both
  directions — not just that the dashboard loads (a stale one-way
  tunnel can still serve a cached page, which is the failure mode
  this slice catches). Two probes (outbound HTTPS via magicdns +
  inbound nonce POST) plus a round-trip nonce comparison tell the
  operator honestly whether their van is reachable from anywhere.

  Per the doctrine ('must not fail + super intuitive + critical
  infrastructure'): the probes have 10-second timeouts so a stalled
  tunnel never blocks the dashboard; the recovery automation fires
  after 60s and surfaces a plain-English notification instead of
  dropping the operator into a manual recovery; the §8.T.3 wizard-
  advance automation only fires when the global setup stage is at
  'networking' (idempotent — no-ops if the parent wizard already
  advanced it).

Changes

  NEW (4 files):
    - homeassistant/packages/roamcore_tailscale_self_test.yaml
        Inputs (tunnel URL password-mode + running flag + Run now
        button + last_run stamp), 2 shell_command probes (10s
        timeout each), 3 command_line sensors (HTTP code +
        expected/received nonce files), 4 template binary_sensors
        (outbound_ok / inbound_ok / ok / recovery), 1 template
        sensor (status — 6 plain-English branches, zero jargon),
        3 §8 MANDATORY automations: §8.T.1 run (button + stage +
        event triggers, 30s settle, no URL clear), §8.T.2 recovery
        (60s timeout, persistent_notification, no URL clear),
        §8.T.3 wizard advance (only fires when rc_setup_stage ==
        networking, flips to 'map', idempotent).
    - homeassistant/packages/tests/test_tailscale_self_test.py
        Pytest rig (52 tests, all green): YAML parses, every
        required helper / automation / template entity present,
        §8.T.1/§8.T.2/§8.T.3 trigger+action contract verified,
        tunnel URL is mode: password (sensitive), running flag
        defaults off, last_run initial epoch, status template
        covers all 6 (url, running, outbound, inbound, recovery,
        never_run) combos via a pure-function helper that pins
        the YAML strings (catches silent drift), no operator
        jargon in user-facing output (forbidden terms: entity ID
        prefixes, bash, curl, magicdns, .ts.net, tskey-), YAML
        idempotent (PyYAML twice → same dict), shell_command has
        exactly 2 probes, command_line sensors have unique
        unique_ids, §8.T.1 + §8.T.2 do NOT call input_text.set_value
        on the tunnel URL (idempotent retry), rc-entity-naming
        compliance on every entity_id / automation id, IKEA doc
        5-step shape, no SUPERSEDED / Cron-handoff refs in user
        copy.
    - scripts/checks/tailscale-self-test-smoke.sh
        Bash smoke wrapper (10 standalone bash assertions +
        pytest rig): file presence, YAML parse, rc-naming pre-
        check, helpers + automations + templates pre-check, status
        template covers all branches, no operator jargon in status
        copy, secrets-leak check (no tskey- / no IPs), idempotency
        probe (PyYAML twice → same dict + exactly 2 probes), IKEA
        doc 5-step shape + translation table, §8.T.1 + §8.T.2
        wiring verified, pytest rig green.
    - docs/setup/tailscale-self-test.md
        IKEA 5-step user guide: What this is / What you see (6
        plain-English status phrases) / What you do (3 numbered
        steps) / What to do if it goes wrong (3 sub-cases) /
        Useful links + an operator→vanlifer translation table
        (Self-test / Round-trip / Outbound probe / Inbound probe /
        MagicDNS / ACL rules / Tunnel / Tailscale / Two-way /
        Persistent notification / input_button / input_text /
        Template sensor / Automation).

  ADDITIVE EDITS (3 files):
    - connections/remote-access/connection.yml (+45 lines)
        Extended wizard.setup_paths with the self_test entry
        (separate from the four remote-access paths A/B/C/D —
        doesn't pick a path, verifies the chosen path honestly
        works two-way). tier: b, recipe_over, estimated_time,
        requires_inputs, side_effects, setup_notes all
        documented. Path A (tier-a promotion candidate) is still
        wired; this slice is purely additive.
    - homeassistant/packages/roamcore_setup_wizard.yaml (+19 lines)
        New automation.rc_setup_trigger_tailscale_self_test_after_done:
        when rc_remote_access_setup_stage reaches 'tailscale_done',
        fires the 'rc_run_tailscale_self_test' event so the
        self-test package runs its probes. Idempotent: re-firing
        does NOT reset the tunnel URL or wizard progress.
    - scripts/check.sh (+1 line)
        Wire scripts/checks/tailscale-self-test-smoke.sh into the
        core-only chain at a stable slot next to
        remote-access-setup-smoke.sh (no duplicate wiring — the
        in-flight sub-agent's duplicate line was cleaned up).

Verification

  - bash scripts/check.sh --core-only is GREEN on this branch tip:
      • Tailscale self-test smoke: 10/10 PASS
      • Tailscale self-test pytest rig: 52/52 PASS (0.19s)
      • All other 32+ connection manifest smokes: GREEN
      • Connection state field smoke: GREEN
      • PWA install smoke: GREEN (26/26)
      • Hub golden-image smoke: GREEN (10/10)
      • Final summary: 'all requested smoke checks passed'
        (182 PASS markers / 0 FAIL markers across the chain).
  - The 52 pytest tests cover:
      • YAML structural sanity + idempotency.
      • Every required helper / automation / template entity
        present and named per rc-entity-naming.md.
      • §8.T.1 / §8.T.2 / §8.T.3 trigger + condition + action
        contract (including idempotent retry: no input_text.set_value
        on the tunnel URL helper).
      • status template covers all 6 state combinations via the
        pure-function helper pinned to the YAML strings.
      • status output has no operator jargon (entity IDs / bash /
        curl / magicdns / .ts.net / tskey-).
      • secrets-leak grep: no tskey- / no hard-coded IPv4 in the
        package.
      • IKEA doc 5-step shape + opener plain-English + translation
        table present + no SUPERSEDED / Cron-handoff.
      • shell_command probes both have 10s timeout.
      • Wizard advance advances to 'map' (NOT 'done') so it
        doesn't skip the rest of the setup wizard.

Rollback

  Revert this single commit. All changes are additive (no rewrites,
  no deletions). The new helpers / automations / sensors live in a
  brand-new file that nothing else in the repo depends on until
  the parent wizard reaches 'tailscale_done'. Removing the file +
  the bridge automation in roamcore_setup_wizard.yaml + the
  self_test entry in the connection's setup_paths block + the
  check.sh wire restores main to its prior state bit-for-bit.

  No Proxmox / HA / OpenWrt / networking / vmbr0 touched → no
  snapshot required (GOLDEN.md E3 N/A).

GOLDEN.md alignment (re-read 2026-08-10)

  Product principles served (quoted verbatim from GOLDEN.md):
    P1 (Novice-first UX — Apple-grade onboarding. Power users get an
       'Advanced mode.' Fail-softly UI everywhere.):
       The self-test card surfaces one plain-English sentence at a
       time (6 status branches, zero operator jargon); the IKEA doc
       walks the operator through 3 numbered steps; the recovery
       automation fires a plain-English notification instead of an
       error; the running flag defaults off so the dashboard doesn't
       lie about state on first boot.
    P2 (Mission-critical connectivity — LTE primary, Starlink
       failover, Tailscale-like remote access. Network failures must
       not lose data or lock the user out.):
       The two-probe round-trip catches the failure mode where a
       one-way tunnel still serves a cached dashboard (the user's
       exact symptom: 'the page loads but nothing works'); the
       10s probe timeouts + 60s recovery automation ensure a stalled
       tunnel never locks the operator out; the tunnel URL is
       preserved across retries so the operator doesn't have to
       re-type it under stress.

  Engineering principles respected:
    E1 (Customer-facing repo): IKEA doc lives at
       docs/setup/tailscale-self-test.md (public MkDocs surface);
       developer plumbing stays in homeassistant/packages/ + tests/
       + scripts/checks/.
    E2 (Documentation-driven): the MkDocs site picks up the new
       doc; cross-linked from docs/setup/guided-remote-access.md
       and docs/setup/local-access-fallback.md.
    E4 (Git fast mode): direct-to-main push on subagent/<slug>;
       no force-push; no PR review chain. This recovery commit
       completes the slice on the in-flight branch.
    E7 (Naming follows docs/reference/rc-entity-naming.md): every
       new entity_id starts with 'rc_tailscale_self_test_' or
       'rc_tailscale_run_self_test'; verified by 4 dedicated pytest
       tests.

  Anti-patterns explicitly avoided (the GOLDEN.md ❌ list):
    ❌ Hand-configuring Victron instead of using capability
       discovery → N/A, no Victron changes.
    ❌ Touching vmbr0 without explicit Bernard override → N/A, pure
       Home Assistant package YAML + IKEA doc.
    ❌ Committing secrets or HA tokens to the repo → verified: no
       tskey-, no hard-coded IPs, no PII; the tunnel URL is
       'mode: password' and operator-entered; the secrets-leak
       check is in the smoke and runs in CI.
    ❌ Adding 'advanced' features before the novice path is solid
       → the entire slice IS the novice path; no advanced mode
       hooks touched.
    ❌ Wide PRs that mix networking + UI + infra changes → only
       files in scope were touched; 4 NEW + 3 MODIFIED, all in
       the connections/remote-access + packages + docs tree.
    ❌ Pulling in unrelated project context → no other projects
       referenced; RoamCore-only.
    ❌ Putting internal engineering logs on the public GitHub →
       the IKEA doc opens with a plain-English sentence and has
       an operator→vanlifer translation table at the bottom.

  Tier discipline: kept tier: b on the connection manifest entry;
  Path A (tier-a promotion candidate) is the wired-up one; this
  slice is purely additive and does not promote any tier.

Doc-target check: docs/setup/tailscale-self-test.md (IKEA 5-step,
plain English, no jargon, with translation table). Confirmed in
docs/ tree, not in memory/ or Cron-handoff/.

Co-authored-by: Bernard <bernard@roamcore.local>
Context: Wave 9 #119b — Phase 2 canonical vehicle model mapping layer.
The schema primitive (commit d5138ed, #119a) shipped
`connections/_schema/canonical_capabilities.json` + the validator in
`vehicle_model.py`. This slice is the mapping layer (#119b): vendor
entity IDs → canonical capability IDs. The mapping layer is what the
dashboard generator (#119c) and the verification framework (#119d)
consume next; without it, RoamCore cannot decide which vendor device
"is" the leisure battery.

GOLDEN.md alignment:
- Product principles served:
  P2 (Mission-critical connectivity — "the same dashboard card always
  means the same thing" is the abstraction that makes connectivity
  failures diagnosable),
  P3 (Victron-centric power MVP — auto-discovery requires a confident
  vendor→canonical mapping),
  P6 (OpenClaw first-class citizen — agent queries read canonical
  names, not vendor IDs).
- Engineering principles respected:
  E1 (Customer-facing repo — IKEA doc, no internal jargon in user copy),
  E2 (Documentation-driven — MkDocs nav entry + canonical reference doc),
  E6 (HACS-friendly layout — pure stdlib module fits in custom_components),
  E7 (Naming follows `docs/reference/rc-entity-naming.md` — re-read
  fresh, honored).
- Anti-patterns explicitly avoided:
  ❌ "Hand-configuring Victron instead of using capability discovery"
     (this slice IS the auto-discovery mapping),
  ❌ "Wide PRs that mix networking + UI + infra changes" (mapping
     logic only, no UI changes),
  ❌ "Committing secrets or HA tokens to the repo" (no tokens, no IPs,
     no PII),
  ❌ "Pulling in unrelated project context" (RoamCore-only),
  ❌ "Putting internal engineering logs on the public GitHub"
     (everything user-facing lives under `docs/`).

Changes:
- NEW `homeassistant/custom_components/roamcore/capability_mapping.py` (≈451 LOC)
  Pure stdlib + json module. Exports: `load_mapping_rules`,
  `map_entity_to_capability` (returns `(canonical_id, info)` with
  confidence + reason + candidates), `build_capability_map`
  (`{canonical_id: vendor_entity_id}` with alphabetical tie-break),
  `unmapped_entities` (sorted leftovers). Confidence ranking:
  exact rule = 1.0, example_sources match = 0.9, fuzzy suffix match =
  0.6 (flagged low_confidence). `AmbiguousMappingError` on equal
  high-confidence rule hits; `MappingRuleError` on malformed rules.
  Names per `docs/reference/rc-entity-naming.md`; re-declares
  `FORBIDDEN_VENDOR_TOKENS` with a comment pointing at
  `vehicle_model.py` as the canonical source.
- NEW `homeassistant/custom_components/roamcore/tests/test_capability_mapping.py` (≈819 LOC, 53 pytest tests)
  Mirrors the import-by-file-path pattern from `test_vehicle_model.py`
  so pytest never pulls in HA runtime. Covers load_mapping_rules
  accept/reject paths, map_entity_to_capability exact-rule matches
  for every documented vendor (victron / starlink / peplink / teltonika
  / generic), example_sources matches, fuzzy suffix matches with
  candidates, ambiguous-match raise, input validation, build_capability_map
  confident subset, alphabetical tie-break, deterministic unmapped
  leftovers, end-to-end six-category mapping for a typical van setup,
  and a cross-cutting rc-naming + no-vendor-token sweep.
- NEW `scripts/checks/capability-mapping-smoke.sh` (≈227 LOC)
  bash strict-mode smoke. AST-parses the module, parses + validates
  the rules JSON, runs the pytest rig, sweeps the rules file for
  rc-naming + no-vendor-tokens, runs the end-to-end canonical map
  against the shipped schema, and asserts the IKEA doc has the 5
  required sections. Wired into `scripts/check.sh`.
- +9 lines `scripts/check.sh` — `run_if_present` wire-up + 6-line
  explanatory comment block.
- +1 line `mkdocs.yml` — nav entry under "Technical (advanced)".
- NEW `connections/_schema/capability_mapping_rules.json` (63 LOC)
  14 explicit vendor-entity → canonical-capability rules covering
  the documented vendor prefixes (victron `vt_`, starlink `dish_`,
  peplink `pep_`, teltonika `rut_`) + generic fallback patterns
  for the 6 default capability categories.
- NEW `docs/reference/rc-capability-mapping.md` (≈95 LOC)
  IKEA-style 5-step user-facing doc: opens with a plain-English
  one-sentence intro, §1 what it does / §2 what you see /
  §3 what you do / §4 what to do if it goes wrong / §5 useful
  links, plus an operator→vanlifer translation appendix.

Verification:
- `bash scripts/check.sh --core-only` exits 0 (GREEN) on the
  branch tip.
- `bash scripts/checks/capability-mapping-smoke.sh` passes
  (53 pytest tests + 7 inline assertions, all rc-naming compliant).
- `python3 -m pytest homeassistant/custom_components/roamcore/tests/`
  → 96 tests pass (53 new + 43 existing from `test_vehicle_model.py`).
- Canonical mapping verified to NOT leak vendor tokens (end-to-end
  sweep against the shipped schema + rules + typical van setup).
- rc-entity-naming compliance verified (every canonical id starts
  with `rc_`, every `FORBIDDEN_VENDOR_TOKENS` token absent).
- Verification step is real pytest (not a stub).

Rollback: revert this commit; remove the `run_if_present` block
in `scripts/check.sh`; revert the `mkdocs.yml` nav entry.

User-facing: Tells RoamCore "this Victron battery sensor IS the same
battery as the Renogy battery sensor IS the same battery as any
other brand" — so your dashboard always shows the same battery card
no matter which brand you plug in.
…framework.

The schema primitive (commit d5138ed, #119a) shipped
`connections/_schema/canonical_capabilities.json` + the validator in
`vehicle_model.py`. This slice is the verification framework (#119d):
post-install verify per Directive Rule 4 ("Don't mark successful if
automated action can't be verified"). Sibling slice to the mapping
layer (#119b, landed at 1a98605) and the dashboard generator (#119c),
all three consuming the same schema primitive.

Per the slice spec: successful install != working connection. Post-
install checks: voltage available, values plausible, SoC present
where expected, data updates, device reconnects after restart. This
slice turns "the tile is showing" into "the tile is showing REAL
DATA that we just verified".

GOLDEN.md alignment:
- Product principles served:
  P1 (Novice-first UX — Apple-grade onboarding. Power users get an
     "Advanced mode." Fail-softly UI everywhere.) — the verification
     framework surfaces plain-English `reason` + `recovery_hint`
     strings on every failure mode. The user always gets one sentence
     in operator→vanlifer language, never a stack trace.
  P2 (Mission-critical connectivity — LTE primary, Starlink failover,
     Tailscale-like remote access. Network failures must not lose
     data or lock the user out.) — verification catches the silent-
     failure mode (tile shows stale data + a "verified fresh" badge
     that lies) before the user trusts bad data.
  P6 (OpenClaw as a first-class citizen — every visible data point
     is also readable from OpenClaw. "Chat with the van.") — OpenClaw's
     "is the battery OK?" answer requires verified, fresh data, not
     stale values.
- Engineering principles respected:
  E1 (Customer-facing repo — RoamCore GitHub is what the world sees.
     Internal engineering logs, sensitive infra details, and
     unfiltered agent run transcripts stay local (memory/roamcore/).)
     — IKEA doc at docs/reference/rc-verification.md, no internal
     jargon in user copy; the translation table at §5 maps every
     operator term to its vanlifer equivalent.
  E2 (Documentation-driven — MkDocs site is the public surface.
     docs/ is the source of truth for users. Runbooks/scripts are
     sanitised.) — mkdocs.yml nav entry added; reference doc
     follows the established 5-step IKEA shape.
  E7 (Naming follows docs/reference/rc-entity-naming.md) — re-read
     fresh; every canonical id flowing through the framework starts
     with rc_ and contains no vendor tokens (enforced by lazy-loaded
     FORBIDDEN_VENDOR_TOKENS from vehicle_model.py + the rc-naming
     sweep in the bash smoke).
- Anti-patterns explicitly avoided:
  ❌ "Hand-configuring Victron instead of using capability discovery"
     (verification is the missing half of capability-driven auto-
     discovery — it confirms the discovered capability is actually
     working, which makes auto-discovery MORE honest).
  ❌ "Pulling in unrelated project context" (RoamCore-only; no
     cross-project imports, no memory/, no Cron-handoff/ refs).
  ❌ "Wide PRs that mix networking + UI + infra changes"
     (verification contract only — no UI runtime changes; no
     Proxmox / OpenWrt / HA integration; no networking).
  ❌ "Committing secrets or HA tokens to the repo" (no tokens, no
     IPs, no PII; verified by the secrets-leak-style grep in the
     bash smoke + the rc-naming sweep).
  ❌ "Touching vmbr0 without explicit Bernard override" (N/A — no
     Proxmox / OpenWrt / networking change in this slice).
  ❌ "Adding 'advanced' features before the novice path is solid"
     (this slice IS the novice path's safety net — every check
     surfaces a one-sentence plain-English message).
  ❌ "Putting internal engineering logs on the public GitHub" (the
     IKEA doc opens with a plain-English sentence and lives at
     docs/reference/rc-verification.md; developer plumbing stays in
     homeassistant/custom_components/roamcore/ + tests/ + scripts/).

Changes (6 files, all additive, no rewrite):
- NEW homeassistant/custom_components/roamcore/verification.py (~470 LOC)
  Pure-stdlib + json verification framework. No HA imports. Public
  surface: VerificationTarget + VerificationResult dataclasses,
  verify_connection() top-level entry, verify_capability_map() batch
  helper, assert_rc_prefix() + assert_no_vendor_tokens() guards.
  Implements all 5 verification checks:
    1. value_in_range — most recent sample within expected_range
       (derived from canonical capability's unit + device_class when
       not supplied: battery % → 0..100, voltage → 0..60, current →
       -500..500, power → -5000..5000, temperature → -50..80,
       latitude → -90..90, longitude → -180..180).
    2. recent_update — at least one sample in the last 60s
       (DEFAULT_FRESHNESS_SECONDS).
    3. data_plausible — no impossible sample-to-sample jumps (default
       rule: any delta > 50% of expected range in < 60s fails).
    4. restart_resilience — restart_count >= 0 AND if
       last_disconnect_at is set, samples exist AFTER it.
    5. vendor_match — entity_id's prefix matches one of the canonical
       example_sources families (substring match, not regex).
  CHECK_ORDER is stable and exported so tests can assert
  first-failing-check-wins aggregation.
- NEW homeassistant/custom_components/roamcore/tests/test_verification.py
  (~850 LOC, 52 pytest tests, all PASS in 0.07s). Loads both
  verification.py and vehicle_model.py by file path (mirrors the
  test_vehicle_model.py pattern) so pytest doesn't pull in the HA
  runtime. Covers every check's pass + fail paths + skip paths,
  verify_connection aggregation, verify_capability_map with mock
  5-capability van setup, rc-entity-naming belt-and-braces guards,
  anti-slop scan of every reason + recovery_hint string, plain-
  English shape (every reason is one clause + starts naturally),
  recovery_hint length cap (<=30 words), and 100x idempotency sweep.
- NEW scripts/checks/verification-framework-smoke.sh (~250 LOC, 8+
  inline bash assertions + pytest rig). Mirrors the convention in
  scripts/checks/<name>.sh: bash strict mode, repo-local only,
  plain-English summary at exit 0 / non-zero exit. Asserts: module
  parses as Python, module is pure stdlib + json (no HA imports via
  AST scan), pytest rig reports >=30 tests all PASS, shipped schema
  canonical ids all match rc-naming + contain no vendor tokens,
  runtime test rig ids same, IKEA doc has 5+ numbered sections, IKEA
  doc is jargon-free (operator->vanlifer table enforced; the section
  5 translation table is excluded because that's where operator-speak
  appears verbatim by design), and every reason/recovery_hint string
  produced by the framework passes the anti-slop guard.
- MODIFIED scripts/check.sh (+1 run_if_present line). Adds the
  verification framework smoke as an additive step right after the
  capability mapping smoke, both of which sit after the canonical
  capabilities schema primitive smoke.
- NEW docs/reference/rc-verification.md (~110 LOC, IKEA 5-step shape).
  Opens with one plain-English sentence ("After RoamCore adds a
  device, it checks that the device is actually talking..."), section 1
  What this is / section 2 What you see / section 3 What you do /
  section 4 What to do if it goes wrong / section 5 Useful links
  (operator->vanlifer cheat sheet). Every sentence in sections 1-4
  passes the operator->vanlifer translation table (verified by the
  bash smoke).
- MODIFIED mkdocs.yml (+1 line under Technical (advanced) > Reference).
  Adds the "Post-install check: reference/rc-verification.md" nav
  entry so the doc is discoverable on the public MkDocs site.

Verification: bash scripts/check.sh --core-only -> GREEN (exit 0);
pytest 52 tests PASS in 0.07s; bash smoke 8+ assertions PASS in <1s;
real pytest (not stub); all 5 verification checks implemented +
tested; all reason/recovery_hint strings pass the operator->vanlifer
translation table; rc-entity-naming compliance verified against the
shipped schema (15 canonical ids) + the test rig's runtime ids.

Rollback: revert commit; remove the run_if_present line in
scripts/check.sh; remove the mkdocs.yml nav entry.

User-facing: Checks that each device is actually sending fresh,
sensible data — not just that the card is on the screen — and tells
you honestly in plain English if anything looks wrong, with one
sentence about what to do next.

import re
from pathlib import Path
from typing import Any
from __future__ import annotations

import importlib.util
import json
"sensor.aaa_unknown": {},
"sensor.vt_battery_soc_percent": {},
}
result = build_capability_map(entities, _SHIPPED_RULES_DOC, caps)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant