Skip to content

test(server): stabilize async root span assertions - #2583

Open
pimlock wants to merge 3 commits into
mainfrom
stabilize-reconcile-span-test/pm
Open

test(server): stabilize async root span assertions#2583
pimlock wants to merge 3 commits into
mainfrom
stabilize-reconcile-span-test/pm

Conversation

@pimlock

@pimlock pimlock commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Stabilizes async OpenTelemetry root-span assertions that can snapshot the in-memory exporter after child spans finish but before the root span is observable.

The reported linux-arm64 failure showed completed driver and store child spans without the expected reconcile.sandboxes root span: Actions failure.

A follow-up linux-amd64 run exposed the same symptom in the provider refresh test because its explicit current-span handle remained alive while the assertion polled: Actions failure.

This uses the same bounded polling pattern already used by the disconnected watch-span test added in #2582.

Related Issue

No issue required: this is an obvious localized CI test flake.

Changes

  • Wait up to five seconds for the reconciliation root span, polling every 10 ms before taking the assertion snapshot.
  • Audit the remaining async root-span tests and apply the same pattern to driver watch events and provider refresh ticks.
  • Explicitly release the provider refresh current-span handle after recording attributes so the root span can close before exporter polling.
  • Leave exporter tests that already explicitly drop their span handles unchanged.

Testing

  • mise run pre-commit passes
  • Focused async root-span tests pass
  • mise run test passes, including all 1,228 server tests
  • mise run ci completed lint and compile checks, but a supervisor-process test binary hung locally after its visible tests passed; stopped after five minutes
  • E2E tests not applicable for telemetry test stabilization

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs not applicable for telemetry test stabilization

pimlock added 2 commits July 31, 2026 18:40
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant