Skip to content

CI: Emit JUnit from test runners, pin their exec bounds - #1687

Merged
Max Smythe (maxsmythe) merged 1 commit into
agent-substrate:mainfrom
stshrive:dev/emit-junit
Sep 23, 2026
Merged

Max Smythe (maxsmythe) merged 1 commit into
agent-substrate:mainfrom
stshrive:dev/emit-junit

Conversation

@stshrive

@stshrive Steven Shriver (stshrive) commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Today CI depends on process exit codes. This leads to a few issues with reporting results. For example, a green build cannot be distinguished from one where no tests ran at all; a -run filter that matches nothing; an e2e suite short-circuiting without --e2e; or a testcontainer skip: all currently exit 0.

This change adds gotestsum as a pinned tool module allowing opt-in for JUnit reporting in both test runners by using E2E_JUNIT_FILE and ROOT_JUNIT_FILE respectively. When left unset, the runners exec go test directly and gotestsum is never built leaving local runs unchanged for now.

Additionally, this change pins two execution bounds that go test would otherwise infer. Both remain overridable via environment variables as listed below.

  • -p: system defaults didn't align with cluster capacity so pinning the value avoids over allocation. (override with E2E_PARALLELISM).

  • -timeout: Suite defaults conflicted with some E2E test timings causing some tests that are expected to timeout at the same time to abort rather than fail as expected. Setting an 30m default avoids the conlficting timeouts producing clean failure signals (override with E2E_TIMEOUT).

Fixes #1685

Comment thread hack/run-e2e.sh Outdated
Today CI depend on process exit codes. This leads to a few issues
with reporting results. For example, a green build cannot be
distinguished from one where no tests ran at all; a `-run` filter
that matches nothing; an e2e suite short-circuiting without `--e2e`;
or a testcontainer skip: all currently exit 0.

This change adds gotestsum as a pinned tool module allowing opt-in
for JUnit reporting in both test runners by using `E2E_JUNIT_FILE`
and `ROOT_JUNIT_FILE` respectively. When left unset, the runners exec
`go test` directly and gotestsum is never built leaving local runs
unchanged for now.

Additionally, this change pins two execution bounds that `go test`
would otherwise infer. Both remain overridable via environment
variables as listed below.

  `-p`: system defaults didn't align with cluster capacity so
  pinning the value avoids over allocation. (E2E_PARALLELISM).

  `-timeout`: Suite defaults conflicted with some E2E test timings
  causing some tests that are expected to timeout at the same time
  to abort rather than fail as expected. Setting an 30m default
  avoids the conlficting timeouts producing clean failure signals.
  (E2E_TIMEOUT).

@maxsmythe Max Smythe (maxsmythe) left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxsmythe
Max Smythe (maxsmythe) added this pull request to the merge queue Sep 23, 2026
Merged via the queue into agent-substrate:main with commit 898f6e6 Sep 23, 2026
11 checks passed
@stshrive
Steven Shriver (stshrive) deleted the dev/emit-junit branch September 23, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Couldn't load reviewers.

Assignees

Couldn't load assignees.