STAC-25500 Port the DEB signing and pre-release publishing lane to GitHub Actions - #455
Conversation
…tion Addresses review feedback on PR #455. [P1] The signing job installed the publisher with `gem install deb-s3`, resolving the latest code at run time into a job that then executes it with the package signing key and the pre-release AWS credentials in scope. GitLab never did this: `sign_deb` ran in a pinned image with deb-s3 already baked in, so the runtime resolve was a regression introduced by the port. deb-s3 and its full runtime dependency tree are now pinned in `.github/deb-s3-gems.sha256` and installed by `.github/scripts/install-deb-s3.sh`, which fetches each gem at its exact version and verifies it against the SHA256 RubyGems publishes for that release before anything is installed or executed. The manifest covers nine gems. base64, bigdecimal and logger are deliberately excluded: aws-sdk-core requires them at ">= 0" and they are Ruby default gems supplied by the distribution's own ruby package, so pinning them would force a native build for no supply-chain gain. The script also links the canonical executable when RubyGems installs a versioned binstub, since publish_package.sh invokes deb-s3 by bare name, and ends with a `deb-s3 help` smoke check that activates the whole pinned set so a missing or incompatible gem fails at install time rather than midway through publishing. [P2] publish-install-script depended only on generate-install-script, so it could overwrite the public install.sh even when the build, signing or apt upload had failed. GitLab's pre_release_deb required sign_deb. It now requires both generate-install-script and sign-and-publish-deb, restoring that release boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tion Addresses review feedback on PR #455. [P1] The signing job installed the publisher with `gem install deb-s3`, resolving the latest code at run time into a job that then executes it with the package signing key and the pre-release AWS credentials in scope. GitLab never did this: `sign_deb` ran in a pinned image with deb-s3 already baked in, so the runtime resolve was a regression introduced by the port. deb-s3 and its full runtime dependency tree are now pinned in `.github/deb-s3-gems.sha256` and installed by `.github/scripts/install-deb-s3.sh`, which fetches each gem at its exact version and verifies it against the SHA256 RubyGems publishes for that release before anything is installed or executed. The manifest covers nine gems. base64, bigdecimal and logger are deliberately excluded: aws-sdk-core requires them at ">= 0" and they are Ruby default gems supplied by the distribution's own ruby package, so pinning them would force a native build for no supply-chain gain. The script also links the canonical executable when RubyGems installs a versioned binstub, since publish_package.sh invokes deb-s3 by bare name, and ends with a `deb-s3 help` smoke check that activates the whole pinned set so a missing or incompatible gem fails at install time rather than midway through publishing. [P2] publish-install-script depended only on generate-install-script, so it could overwrite the public install.sh even when the build, signing or apt upload had failed. GitLab's pre_release_deb required sign_deb. It now requires both generate-install-script and sign-and-publish-deb, restoring that release boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
f31b3c3 to
ed3b944
Compare
2d5cbce to
61e2bce
Compare
…tion Addresses review feedback on PR #455. [P1] The signing job installed the publisher with `gem install deb-s3`, resolving the latest code at run time into a job that then executes it with the package signing key and the pre-release AWS credentials in scope. GitLab never did this: `sign_deb` ran in a pinned image with deb-s3 already baked in, so the runtime resolve was a regression introduced by the port. deb-s3 and its full runtime dependency tree are now pinned in `.github/deb-s3-gems.sha256` and installed by `.github/scripts/install-deb-s3.sh`, which fetches each gem at its exact version and verifies it against the SHA256 RubyGems publishes for that release before anything is installed or executed. The manifest covers nine gems. base64, bigdecimal and logger are deliberately excluded: aws-sdk-core requires them at ">= 0" and they are Ruby default gems supplied by the distribution's own ruby package, so pinning them would force a native build for no supply-chain gain. The script also links the canonical executable when RubyGems installs a versioned binstub, since publish_package.sh invokes deb-s3 by bare name, and ends with a `deb-s3 help` smoke check that activates the whole pinned set so a missing or incompatible gem fails at install time rather than midway through publishing. [P2] publish-install-script depended only on generate-install-script, so it could overwrite the public install.sh even when the build, signing or apt upload had failed. GitLab's pre_release_deb required sign_deb. It now requires both generate-install-script and sign-and-publish-deb, restoring that release boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ed3b944 to
e3a578b
Compare
Rebase regression fixed — amd64 gate is green
When this branch was rebased onto the updated #454, the conflict in Before ( After ( That now matches #454 exactly. Verified structurally too — Remaining red is one external gap, not this PRThe only failing jobs left on this branch are the two arm64 container image builds:
Both die inside The message is misleading. The real cause is that the lean arm64 runner image does not ship The timing confirms it: arm64 "skipped" 1105 documents in 830 ms (~0.75 ms each, too fast to have forked a process) against ~32 ms per document on amd64. Fix is open as stackstate-ci-images#16 (STAC-25559). Once that image is built and the |
…itHub Closes the last two GitLab jobs with no GitHub counterpart: sign_deb and pre_release_deb (.gitlab-ci-agent.yml lines 604 and 635). Security prerequisite, not cleanup ---------------------------------- sign_debian_package.sh called printenv unconditionally. On GitLab that dumped the GPG private key and its passphrase into the job log; stackstate-agent is a PUBLIC repo, so on GitHub that log is world-readable. It also left the exported private key in the checkout as gpg_private.key, where any later artifact upload would collect it. Both are removed before any signing secret is wired in. Key setup now happens in an ephemeral GNUPGHOME created with mktemp and mode 700, removed by an EXIT trap that also kills the gpg-agent so a preset passphrase cannot outlive the job on a reused runner. The passphrase reaches gpg through a mode-600 file inside that directory rather than argv. Fixes a latent signing bug -------------------------- The old preset step interpolated an unquoted command substitution into a single gpg-preset-passphrase call. A key exposes one keygrip per primary and subkey, so with a signing subkey the second keygrip was passed as a stray argument and never presetted, leaving signing able to block on a pinentry prompt no CI runner can answer. Each keygrip is now presetted individually. Verified against a throwaway key: two keygrips, both presetted. Shared setup ------------ sign_debian_package.sh and publish_package.sh need the same key in the same state but are separate processes, and on GitHub may be separate steps, so neither can rely on a keyring the other left behind. The setup moves into gpg_signing_setup.sh, sourced by both. gpg-preset-passphrase is auto-detected across /usr/lib/gnupg2, /usr/lib/gnupg, /usr/libexec and PATH, and the script fails loudly rather than silently skipping the preset when it is absent. Both scripts now fall back from CI_PROJECT_DIR to GITHUB_WORKSPACE, and publish_package.sh falls back from CI_COMMIT_REF_NAME to GITHUB_REF_NAME, so the apt codename stays the release branch exactly as it was on GitLab. Workflow -------- sign-and-publish-deb downloads both architecture artifacts and runs a single deb-s3 upload. GitLab fanned this out per architecture, so two jobs rewrote the same apt index concurrently; collecting both first removes that race. The install script is split in two. generate-install-script runs inv release.generate-install -t inside the build container, because the task collection imports python-gitlab and the GitHub and Datadog API helpers and only loads in the conda environment, and asserts the rendered script contains no None.s3.amazonaws.com from an unset bucket variable. publish-install-script then uploads it, so the container job never holds a credential. Both publishing jobs are gated on push and bound to the agent-pre-release environment, which carries the deployment branch rule, the signing secrets and the AWS role from STAC-25545. Pull requests cannot reach them. Validated: shellcheck -x clean on all three scripts; gpg_signing_setup exercised against a generated throwaway key covering explicit override, PATH auto-detection and the missing-binary failure path; actionlint clean apart from the pre-existing self-hosted runner-label notices; zizmor reports no findings. Blocked until the agent-pre-release environment, its four SIGNING_* secrets and AGENT_PRERELEASE_ROLE_ARN exist. Refs STAC-25546, STAC-25545 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tion Addresses review feedback on PR #455. [P1] The signing job installed the publisher with `gem install deb-s3`, resolving the latest code at run time into a job that then executes it with the package signing key and the pre-release AWS credentials in scope. GitLab never did this: `sign_deb` ran in a pinned image with deb-s3 already baked in, so the runtime resolve was a regression introduced by the port. deb-s3 and its full runtime dependency tree are now pinned in `.github/deb-s3-gems.sha256` and installed by `.github/scripts/install-deb-s3.sh`, which fetches each gem at its exact version and verifies it against the SHA256 RubyGems publishes for that release before anything is installed or executed. The manifest covers nine gems. base64, bigdecimal and logger are deliberately excluded: aws-sdk-core requires them at ">= 0" and they are Ruby default gems supplied by the distribution's own ruby package, so pinning them would force a native build for no supply-chain gain. The script also links the canonical executable when RubyGems installs a versioned binstub, since publish_package.sh invokes deb-s3 by bare name, and ends with a `deb-s3 help` smoke check that activates the whole pinned set so a missing or incompatible gem fails at install time rather than midway through publishing. [P2] publish-install-script depended only on generate-install-script, so it could overwrite the public install.sh even when the build, signing or apt upload had failed. GitLab's pre_release_deb required sign_deb. It now requires both generate-install-script and sign-and-publish-deb, restoring that release boundary. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolving the rebase conflict against the rewritten base took this branch's whole copy of build-deb.yml, which predated the exception work on STAC-25457. That silently reverted two lines, so the gate ran here with no exceptions loaded and reported all 24 findings as unmanaged. Restores the quay LOCAL_IMAGE name the evaluator matches exceptions on, and the exceptions-path input. The workflow diff against the base branch is now additions only. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
61e2bce to
19fa3b9
Compare
1a2d643 to
185e763
Compare
Closes the last two GitLab jobs on this repo with no GitHub counterpart:
sign_debandpre_release_deb(.gitlab-ci-agent.ymllines 604 and 635). Stacked on #454.Security prerequisite, not cleanup
sign_debian_package.shcalledprintenvunconditionally. On GitLab that dumped the GPG private key and its passphrase into the job log.stackstate-agentis a public repo, so on GitHub that log is world-readable. It also left the exported private key in the checkout asgpg_private.key, where any later artifact upload would collect it.Both are removed before any signing secret is wired in, as STAC-25500 requires. Key setup now happens in an ephemeral
GNUPGHOME(mktemp, mode 700) removed by anEXITtrap that also kills thegpg-agent, so a preset passphrase cannot outlive the job on a reused runner. The passphrase reaches gpg through a mode-600 file, never argv.Fixes a latent signing bug
The old preset step was:
A key exposes one keygrip per primary and subkey. Unquoted, that substitution expands to multiple words, so with a signing subkey the second keygrip became a stray argument and was never presetted — leaving signing able to block on a pinentry prompt no CI runner can answer. Each keygrip is now presetted individually.
Verified against a generated throwaway key: two keygrips, both presetted, passphrase delivered on stdin,
GNUPGHOMEremoved by the trap, nothing left in the workspace.Shared setup
The two scripts need the same key in the same state but are separate processes, and on GitHub may be separate steps, so neither can rely on a keyring the other left behind. The setup moves into
gpg_signing_setup.sh, sourced by both.gpg-preset-passphraseis auto-detected across/usr/lib/gnupg2,/usr/lib/gnupg,/usr/libexecandPATH(the path differs by distro and release), and it fails loudly rather than silently skipping the preset when absent.Both scripts fall back
CI_PROJECT_DIR→GITHUB_WORKSPACE, andpublish_package.shfalls backCI_COMMIT_REF_NAME→GITHUB_REF_NAME, so the apt codename stays the release branch exactly as on GitLab.Workflow
sign-and-publish-debdeb-s3 upload. GitLab fanned this out per arch, so two jobs rewrote the same apt index concurrently — collecting both first removes that race.generate-install-scriptinv release.generate-install -tinside the build container: the task collection importspython-gitlabplus the GitHub/Datadog API helpers and only loads in the conda env. Asserts the rendered script contains noNone.s3.amazonaws.comfrom an unset bucket variable —generate_installreads three bucket vars and silently rendersNonefor any that are missing.publish-install-scriptBoth publishing jobs are gated on
pushand bound to theagent-pre-releaseenvironment, which carries the deployment branch rule, the signing secrets and the AWS role. Pull requests cannot reach them.Blocked on
This lane cannot go green until these exist — the code is ready and reviewable now:
agent-pre-releasewith a deployment branch rule for the release branch.SIGNING_PUBLIC_KEY,SIGNING_PRIVATE_KEY,SIGNING_PRIVATE_PASSPHRASE,SIGNING_KEY_ID.AGENT_PRERELEASE_ROLE_ARN, from StackVista/terraform-infra#76 (STAC-25545).Validation
shellcheck -x— clean on all three scripts.gpg_signing_setupexercised against a real generated key: explicit override,PATHauto-detection, and the missing-binary failure path (exits 1 with a clear message).actionlint— clean apart from the pre-existing self-hostedrunner-labelnotices.zizmorv1.28.0 —No findings to report.needs:references resolve.https://stackstate.atlassian.net/browse/STAC-25500
Refs https://stackstate.atlassian.net/browse/STAC-25546 and https://stackstate.atlassian.net/browse/STAC-25545