test: add core component acceptance infrastructure - #841
Conversation
kvinwang
changed the title
7a21a97 to
313efb9
Compare
kvinwang
changed the title
This reverts commit 7a544ad.
Use the main listener's existing /health endpoint for liveness and the authenticated Admin root for the dashboard. Do not require a product-only /health/dashboard alias invented by the acceptance fixture.
TC-GOS-OBSERVABIL-003 already asserted that a peer with no handshake reaches a forced refresh, but it drove the checker with an uneven clock (1000 1010 1191 ...). That 181s jump lets the staleness deadline and the periodic refresh land on separate ticks, which is not how the service runs: the loop sleeps 10s and HANDSHAKE_TIMEOUT equals REFRESH_INTERVAL, so the two deadlines collide every cycle and the periodic refresh clears the staleness timer 10s before it can expire. Add two scenarios on a uniform 10s clock across 45 ticks: - steady_no_handshake: a peer that never handshakes must still reach --force. Only --force rebuilds a tunnel whose config is unchanged, since gateway setup returns early otherwise. - steady_force_rate_limit: with the gateway unreachable, --force must be capped at one attempt per 180s window rather than issued every tick, which would bounce the interface and re-request certificates 45 times. Both were replayed against three checker revisions. The uneven-clock scenario cannot separate the first two; the new ones can: revision uneven no_handshake steady rate limit master 0 forced (fails) 0 44 forced #945 before review 1 forced (passes) 0 45 forced #945 after review 1 forced (passes) 2 3 forced Make the sleep stub's tick budget a parameter so a scenario can run long enough to cross several refresh windows, and require callers to supply more clock values than ticks: when the file drains the stub returns a far-future sentinel whose jump forces a refresh no real deployment would perform, which is what made the checker look correct here. case.md described the old behavior as intended ("A refresh resets the stale timer whether the command succeeds or fails"), which is the defect itself. Restate the contract: only an observed handshake clears the timer, and forced refresh is rate limited because it is the expensive path.
Requested coverage: gateway ACME credential rotation (PR #935)PR #935 reworked Proposed cases for the gateway component:
Cases 3 and 8 also serve as the regression tests for #1010 once an automatic CAA reconciliation loop exists (the loop should make the manual Context: #935 (rework), #1008 (best-effort lock), #1009 (old-account deactivation), #1010 (CAA reconciliation loop). |
|
Updated the Gateway upgrade fixture to stop passing the retired |
|
Replaced the incomplete ZT-domain setup with the production certificate flow. Candidate Gateway fixtures now run isolated Pebble and mock Cloudflare DNS services, configure Certbot through the Admin API, create the DNS credential, register the ZT domain, force certificate issuance, and wait until the certificate is loaded before exercising SNI routing. This no longer depends on Checks run:
|
The checker is no longer wg-checker.sh; it is dstack-util's `gateway-checker` subcommand behind dstack-gateway-checker.service. This case installed the shell script into the guest and pinned its sha256, so it could not run at all against the new image. The old driver derived the refresh matrix (periodic interval, handshake staleness, forced-refresh rate limiting) by injecting a fake clock and fake wg/dstack-util onto PATH. That matrix is now a pure decision function with unit tests in dstack/dstack-util/src/gateway_checker.rs, so reproducing it here would only restate those tests more slowly and less reliably. Dropped. What unit tests cannot reach is the process/systemd boundary, which is what the case now covers: - an app that never enabled dstack-gateway makes the checker exit 0, so Restart=on-failure leaves it alone instead of respawning it every RestartSec on every gateway-less CVM; - a missing gateway app id and a missing gateway URL each exit with EXIT_MISCONFIGURED, read from the product source at run time rather than restated here; - the installed unit is loaded, uses Restart=on-failure, inhibits restart for exactly that code, and runs the subcommand rather than the removed script. The real namespace-isolated WireGuard topology is kept: it exercises the `wg show ... latest-handshakes` shape the checker parses and is unaffected by the rewrite. Also renames the unit in tc-gos-platform-006's graph assertions and in the source inventory and coverage map. Registry: 352 scripted / 352 promoted / 0 findings. Runner unit tests: 35 passed. validate-contracts.py findings are unchanged from the branch point.
|
Added acceptance coverage for merged PR #935 in commit Validation: runner unit tests 35 passed; Python/JSON syntax and |
Summary
This PR is based directly on
masterand contains only core-component acceptance test infrastructure, evidence, and product-PR accounting.dstack-testrunner, registry validation, fixtures, and component harnesses.dstack/**andos/**.Independence and separation invariant
This PR targets
master; it has no dependency on product PR #840 or any split product PR.Its changed paths are limited to:
REUSE.tomldocs/testing/**tools/dstack-test/**docs/test-plans/core-components-full/**Verification
reflects_committed_state=truegit diff --check origin/master...HEAD: passedThe remaining 8 BLOCKED cases require unavailable hardware-backed evidence or GPU capability; they are not test-framework, fixture, documentation, or product-code failures.