STAC-25142 Move the agent CI to GitHub Actions and retire the GitLab pipelines

[LouisParkin]

What

Ports the stackstate-agent GitLab pipeline to GitHub Actions, for amd64 and arm64.

Workflow	Covers
Lint and unit tests	filename linting, inv check-mod-tidy, Zizmor workflow audit, branded + unbranded unit suites
Binary builds	agent and cluster-agent binaries, cluster-agent image build/publish/sign, multi-arch manifest
DEB package build	omnibus DEB, branding verification, agent image build/publish/sign, GPG signing, pre-release apt repository, install script
Go dependency cache image	reusable — bakes the module cache so the heavy jobs start warm instead of re-downloading
Await checks	reusable — holds publication until the other workflows have passed
Cerberus notify	reusable — reports release-branch failures to Slack

Each top-level workflow ends in a CI success (...) aggregate, and those three are the required checks on the base branch. A matrix job cannot be required directly by name, so the aggregate is what branch protection can pin.

Publication is push-only, and gated on the other workflows

Every publishing job is gated on github.event_name == 'push'. A PR builds, smoke-tests and scans the images and the DEB, then throws them away. Jobs needing registry credentials skip on fork PRs.

GitLab ran this as a single pipeline, so publication sat behind every earlier verification job. Three workflows cannot express that with needs:, so await-checks.yml rebuilds the barrier by polling the commit's check runs, failing closed on failure and on timeout. The DEB lane waits for lint and unit tests; the binaries lane waits for those plus the DEB build, which is what strategy: depend on the child pipeline used to enforce. The agent manifest additionally waits for DEB signing, so an image cannot advertise a package that never reached the apt repository.

Nightly

GitLab schedule 20682 builds the release branch every weekday evening (22:00 Europe/Amsterdam; GitHub cron is UTC-only, so the trigger uses 20:00 UTC and drifts an hour in winter). The build and DEB workflows carry an equivalent schedule: trigger so archiving GitLab does not end nightly builds silently. The unit suites are excluded — two ~2h jobs that already run on the PR and on the push.

The nightly does not publish. Image tags are the commit SHA, so a scheduled run on an unchanged branch would re-derive a tag the preceding push already pushed, and the publish action refuses to overwrite. Its value is catching drift in base images and dependencies; Cerberus still reports scheduled failures.

Tag triggers are not carried over

The GitLab rules fired on tags as well as commits. Tags on this repo exist only to satisfy an upstream-merge requirement — nothing builds from them and no release depends on one — so the workflows trigger on push to the release branch, pull_request, schedule and workflow_dispatch only.

GitLab pipelines are removed

The four StackState-owned GitLab pipeline files are deleted, since these workflows replace them. Scheduled runs were the only pipelines GitLab had executed since 27 July, so the nightly above is what was keeping it alive.

Left in place deliberately: .gitlab/ (129 upstream files that ~15 upstream tasks/*.py modules read), .gitlab-scripts/setup_artifact_registry.sh (called by the DEB build here, tasks/omnibus.py and an omnibus software definition), and .ci-builders/. Docs are repointed at the workflows, and UPSTREAM_MERGE.md no longer tells the next merge to rm -rf .github.

The 256 go.mod / go.sum files

One commit, STAC-25466. inv check-mod-tidy is now a gate and the workspace manifests did not survive it. No version was picked by hand; this is what go mod tidy produces.

Validation

All three required aggregates green on 90014e4c, the review-feedback head, including the full arm64 run. Zizmor is clean over the whole tree in both offline and online modes, and actionlint reports nothing beyond the known self-hosted runner-label false positives.

The publish gate was exercised against a stubbed API before it shipped: both checks passing exits zero, a failing check exits non-zero without waiting out the poll, and a check that never appears fails on timeout rather than falling through.

Jira: https://stackstate.atlassian.net/browse/STAC-25142
Sub-tasks merged into this branch: STAC-24889, 25429, 25455, 25457, 25459, 25461, 25464, 25466, 25494, 25498, 25500, 25536, 25575, 25583

[LouisParkin]

Do not merge yet — the DEB signing credentials are only half-provisioned.

pulumi-infra #283 merged and its apply partially succeeded: stackstate-agent now has the agent-pre-release environment (correctly restricted to protected branches) and the AGENT_PRERELEASE_ROLE_ARN variable, but zero secrets in the environment. The pulumi-infrastructure App lacks the environments permission, so it created the environment and then got 403 writing secrets into it (run 31400655652). Tracked as STAC-25571, approved to fix.

sign-and-publish-deb and publish-install-script are gated if: github.event_name == 'push', so nothing here exercises them and this PR stays green — they would fail on the first push to stackstate-7.78.2 after merge.

Safe to merge once the permission is granted and the pulumi apply is re-run green.

[LouisParkin]

Hold cleared — credentials are provisioned.

`agent-pre-release` environment on this repo now has all four `SIGNING_*` secrets, with a protected-branches-only deployment policy, and `AGENT_PRERELEASE_ROLE_ARN` is set. pulumi-infra #283 is merged and applied (run 31400655652).

Remaining blocker on this PR is review only.

[LouisParkin]

Unblocked — this now needs only a review.

pulumi-infra#286 merged and applied, so the release-branch protection reads:

CI success (lint and unit tests)
CI success (DEB package build)
CI success (binary builds)

All three report success on head 3e2a23fe. The two contexts that could never report (the Build … binary names, stale since the build jobs became an arch matrix) are gone, so BLOCKED is now just the one required approving review — enforce_admins is on, so it cannot be bypassed.

[ai-collaboration-app]

Reviewed the ported workflows against the four deleted GitLab pipeline files and the pinned image-pipeline action source. Overall this is a careful port — the fork-PR guards, proxy-defaulted Dockerfile ARGs, full-severity scan gates on locally built images, and the checksum-verified deb-s3 install all land correctly, and the single-arch image publish is actually stricter than GitLab was.

Findings are inline. Two I would treat as blocking:

1. The nightly cannot succeed (build-deb.yml:9). Both push-single-arch and merge-multiarch exit 1 on an existing tag at the pinned SHA, and a scheduled run recomputes the same <sha8> tag the last push published. The PR body states the opposite, so this reads as a genuine misreading of the action rather than an accepted tradeoff. Confirms and extends @LouisLotter's comment.
2. mod-tidy can skip its way to green (lint-and-unit-tests.yml:73) — also @LouisLotter's; I have added why it bites harder given this PR makes that check a gate.

Then three structural ones worth your judgement: the cluster-agent manifest has lost its cross-workflow gates on the DEB build and branding verification (build-binaries.yml:331), the agent manifest has no edge to DEB signing within its own workflow (build-deb.yml:379), and tag-triggered builds are dropped with no mention in the description (build-deb.yml:10).

Three minor: the nightly amd64 cache stagger degenerates to a 15-minute wait, Cerberus loses the commit title on scheduled runs, and one residual unpinned apt-get in the signing job.

On the manifest-gating ones especially — those are a consequence of splitting one GitLab pipeline into three workflows, so there is a real design call to make about whether the gate is worth reconstructing, and you have more context than I do on how the release branch is actually consumed. Happy to be told the risk is acceptable.

Two things I checked that did not turn out to be problems, in case they come up: the dropped floating :stackstate-7.78.2-<arch> tag has no consumers (helm-charts-internal values, its updatecli policy, and beest/helpers/resolve-agent-hashes.sh all resolve sha8 tags), and excluding unit tests from the nightly is faithful — they were when: never on the release branch in GitLab too. All five action tag comments resolve correctly, and the image-pipeline pin is the current merged main tip.

One request: no Zizmor result in the description, and the repo has no zizmor.yml. Also the body cites validation on 3e2a23fe while head is ba25dc9f — the three aggregates are green on ba25dc9f as well, so it is only the body that is stale.

[LouisParkin]

Review feedback addressed — six commits, all threads answered inline.

Commit	Fixes
86ecee68	publication gated on the other workflows' verification (@rb3ckers) — new reusable await-checks.yml, fails closed
59fe9383	Zizmor now a required CI job (@rb3ckers)
f3bb79e1	UPSTREAM_MERGE.md no longer tells the next merge to delete .github (@rb3ckers)
90014e4c	trimmed the cache-metadata commentary (@rb3ckers)
bfe1ba6d	amd64 dependency cache had no leader on scheduled runs
36633e44	Cerberus lost the commit title on scheduled runs

The two blocking bot findings were already fixed in 15681e34 and 2a43e940 before the review landed.

Two left open, both answered inline rather than changed here:

• build-agent-image builds a different image than publish-agent-image publishes, so the scan gate is not running against the shipped artifact. Real, but fixing it properly means changing push-single-arch in the shared image-pipeline, so I would rather follow up than widen this PR. The job itself still has to stay — it is the only image build and scan on a pull request.
• Tag triggers are deliberately not carried over; now stated in the description.

Description refreshed: it no longer claims the nightly publishes, and it covers the publish gate and the tag decision.