Skip to content

Add bin/ci with gh signoff - #7

Open
jeremy wants to merge 11 commits into
mainfrom
local-ci
Open

Add bin/ci with gh signoff#7
jeremy wants to merge 11 commits into
mainfrom
local-ci

Conversation

@jeremy

@jeremy jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member

No cloud CI here, so bin/ci runs the checks locally and signs the commit off on success — the same shape as bcx and highrise.

Why bash, not the Ruby CI class

The app repos use a small Ruby CI class. This repo can't: it exists precisely because a working Ruby is the thing you don't have yet, so its own CI must not require one. house-skills/bin/ci is bash for similar reasons.

Usage

bin/ci              # lint + full build matrix, then gh signoff
bin/ci --lint       # lint only, no Docker — seconds
bin/ci arch         # lint + Arch only
bin/ci arch 2.7.8   # lint + a single build

Only a full run signs off. Anything narrower reports results and explicitly declines to sign — a green tick that covered one platform is worse than no tick.

The lint pass

Cheap, and it catches the two mistakes that otherwise surface ten minutes into a Docker build:

  • Syntax. bash -n over the scripts and every definition. ruby-build sources definitions, so a stray quote is a build failure, not a parse error you'd notice.
  • Checksums. Every install_package URL must carry a #sha256. ruby-build silently skips verification when the checksum is absent, so a missing one downloads and builds whatever it got. Verified this check actually fires by removing a checksum and watching it fail, rather than trusting it to be right.

Shellcheck runs if installed and says so out loud if not, rather than passing silently — a skipped check that reads as green is worse than no check. Gated at --severity=warning: the info tier here is all intentional (ls over find on version-numbered filenames, deliberate word splitting), and gating on it would mean either noisy failures or a scattering of disable comments, both of which train people to ignore the step.

Also in here

Fixing what shellcheck did flag in test/build: two declare-and-assign warnings, plus a comment on the one deliberate unquoted expansion. That one holds two words (--platform linux/amd64) and must split; an array would be the tidier idiom, but expanding an empty array under set -u errors on macOS's Bash 3.2, which this script has to keep working on.

Not included

gh signoff install, which sets branch protection on main and affects everyone on the repo — that's a policy call, not a code change. main is currently unprotected. Say the word and I'll enable it.

Copilot AI review requested due to automatic review settings August 2, 2026 22:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77a25ad488

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
Comment thread bin/ci Outdated
Copilot AI review requested due to automatic review settings August 2, 2026 23:21

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Copilot AI review requested due to automatic review settings August 2, 2026 23:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65e0c116d8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
@jeremy

jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

Pushed 65e0c11 for both Codex findings. Both were the same failure mode — a check reporting success because it inspected nothing — so both now fail loudly instead, with the empty/unparseable cases guarded rather than just the specific triggers.

Also fixed scheduling while in here. JOBS budgeted MAKE_JOBS cores per container; measured against a real run, each averages ~1.0 core, because these builds are mostly single-threaded (miniruby bootstrap and exts.mk generation, the dependency-serialized tail of make, gem install bundler), and 1.9.3 forces make -j1 in its own definition to dodge a parallel-make race. Default is now cores/2: all 12 builds start at once instead of 8, CPU utilisation went from ~25% to 81-93%, and a full bin/ci run dropped from 304s to 234s.

Dogfooded end to end on this branch — 12/12 green, signed off.

@codex review

Copilot AI review requested due to automatic review settings August 2, 2026 23:30

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@jeremy

jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 778e1b1cbe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
Copilot AI review requested due to automatic review settings August 2, 2026 23:40
@jeremy

jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b91da4e923

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
Copilot AI review requested due to automatic review settings August 2, 2026 23:46
@jeremy

jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

Pushed da3df67. Five reports of the same defect meant I was fixing shapes rather than the class — quoted, unquoted, continued, control syntax — so the checksum lint is now inverted: every URL in a definition must carry a digest, not just those on lines that pattern-match an install_package call.

That fails closed. Matching invocations means matching shell syntax, and shell spells the same call unboundedly many ways; each form the pattern doesn't know is a download that goes uninspected, silently reporting success. Checking every URL flags the unanticipated case instead of skipping it. One explicit exemption for GNU's git web view (config.guess/config.sub — moving HEAD, no published digest, never linked into the built Ruby); exempting anything else now requires adding it to that list.

Caught six digest-less forms in testing, two of which review hadn't raised.

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: da3df679a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
Copilot AI review requested due to automatic review settings August 2, 2026 23:51
@jeremy

jeremy commented Aug 2, 2026

Copy link
Copy Markdown
Member Author

@codex review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 15867ede5e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
Copilot AI review requested due to automatic review settings August 2, 2026 23:57

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 925dfcb4fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 3, 2026 00:07

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

jeremy added 9 commits August 2, 2026 19:26
No cloud CI here, so bin/ci runs the checks locally and signs the commit off
on success, the same shape as bcx and highrise.

Bash rather than their Ruby CI class, deliberately: this repo exists because a
working Ruby is precisely what you don't have yet, so its own CI must not need
one. house-skills/bin/ci sets the in-house precedent.

Only a full run signs off. Passing a platform or version narrows the matrix and
explicitly declines to sign — a green tick covering one platform is worse than
no tick.

The lint pass costs nothing and catches the two mistakes that otherwise surface
ten minutes into a Docker build: a syntax error in a definition, since ruby-build
sources these, and an install_package URL with no #sha256, since ruby-build
silently skips verification when the checksum is absent.

Shellcheck runs when present and says so out loud when it isn't, rather than
passing silently. It's gated at warning severity because the info tier here is
all intentional. Fixing what it did flag: two declare-and-assign warnings, and
a note on the one deliberate unquoted expansion, which holds two words and must
split — an array would be tidier but expanding an empty one under set -u breaks
on macOS's Bash 3.2.
…ng builds

Both lint findings are the same failure mode this repo keeps producing: a check
that reports success because it inspected nothing.

sort -V is GNU-only, and macOS — which we explicitly support — ships BSD sort.
The failure doesn't trip set -e in bin/ci because the call sits in a command
substitution inside a `for` list, so definitions() returned empty and both lint
passes sailed over zero files and printed "CI passed". Reproduced with a stub
sort that rejects -V: six definitions silently unchecked, exit 0. Plain sort
here, since ordering is cosmetic when linting, plus a hard guard so an empty
list can never be a pass whatever the cause — wrong directory, failed glob,
broken sort.

test/build had the same -V dependency but failed loudly instead, because a
failing command substitution in a direct assignment *does* trip set -e. Still
broken on macOS, just noisily, so it now detects -V support and falls back.

The checksum lint only matched double-quoted URLs, so a definition written with
single quotes — valid shell — would extract nothing and pass with no digest at
all. Widening the pattern alone would have flagged 1.8.7's `curl '<savannah>'`
calls, which fetch config.guess and have no checksum to carry, so extraction is
now scoped to install_package lines and accepts either quote style. Verified
both directions: single-quoted without a digest fails, with one passes, and the
savannah URLs stay unflagged. Also fails when a definition yields no
install_package URLs at all, since that means the extractor stopped matching
rather than that the file is clean.

Separately, JOBS was budgeting MAKE_JOBS cores per container. Measured against a
real run, each container averages ~1.0 core: these builds are mostly
single-threaded — miniruby bootstrapping and generating exts.mk, the
dependency-serialized tail of make, gem install bundler — with brief parallel
bursts, and 1.9.3 forces make -j1 in its own definition to dodge a race. The old
default left ~75% of the machine idle and could queue the long pole (1.9.3,
~190s vs ~110s) behind short builds. Default to cores/2.
Scoping checksum extraction to lines starting with install_package missed a
call spelled across continued lines: only the first line matched, so a URL on a
continuation went uninspected. Latent rather than live — no definition uses
continuations today — but it's the same vacuous-pass hole one level down, and
the "found no URLs" guard wouldn't have caught it either, since another
well-formed call in the same file keeps the count nonzero.

Fold continuations before matching. Pure bash rather than sed or awk: the usual
line-joining one-liners differ between BSD and GNU, and macOS portability is
what this whole section is about.
`install_package "x" https://…` is valid shell, and requiring quotes meant the
extractor found nothing there — passing a definition with no digest at all
rather than complaining. The "found no URLs" guard doesn't help when another
quoted call in the same file keeps the count nonzero.

This is the third input shape to slip past this check, after single quotes and
line continuations, so stop enumerating quote styles: match the URL itself and
let whitespace or either quote terminate it. One pattern now covers
double-quoted, single-quoted and bare, and the `tr -d` goes away with it.

Scoping to install_package lines stays, and stays load-bearing — over the whole
file this pattern would flag 1.8.7's `curl '<savannah url>'` calls, which fetch
config.guess and carry no checksum by nature.

Verified all four forms with digests pass, each without one fails, and the
savannah URLs stay unflagged.
…ckage

Fifth report of the same defect, so stop fixing shapes. Matching invocations
means matching shell syntax, and shell spells the same call unboundedly many
ways — quoted, unquoted, line-continued, after `then` or `;` or `&&`, inside a
function. Each form the pattern doesn't know is a download that goes
uninspected, and it fails silently: reports success having looked at nothing.
Four such forms turned up in a row, each fix addressing the shape rather than
the class, so the next one was always waiting.

Inverted: every URL in a definition must carry a digest. That fails closed. A
download written in a syntax nobody anticipated is flagged rather than skipped,
and the only way to exempt one is to say so explicitly — right friction, given
an unverified download is what this exists to prevent.

One exemption, GNU's git web view for config.guess/config.sub: a moving HEAD
with no release tarball and no published digest, fetched only to teach ancient
configure scripts about modern architectures, never linked into the built Ruby.

Comment lines are skipped so a URL in prose isn't treated as a download.

Verified against six forms with no digest — including `&&` chaining and a curl
inside a function body, neither of which review had raised — all caught. Real
definitions still pass, savannah URLs stay exempt, commented URLs ignored.
The comment claimed a config.guess/config.sub exemption but the pattern was a
domain wildcard, so any other unverified download from that host would have been
skipped — with the checksummed Ruby URL keeping the count nonzero, silently.
Intent and implementation disagreed, and the implementation was the permissive
one.

Narrowing it first required fixing an extraction bug underneath. Excluding shell
metacharacters from the URL pattern also truncated URLs that legitimately
contain them: both savannah links were being cut at the first semicolon, down to
`?p=config.git`, discarding the `f=config.guess` / `f=config.sub` that says
which file is fetched. Nothing to match on. Metacharacters are now allowed
inside the match and trimmed from the end instead, which is where they actually
signal shell syntax rather than URL content.

Verified: real definitions still pass, a different artifact from the same host
is now caught, so is the same gitweb config.git path requesting another file,
and unquoted URLs trailed by `;` or `&&` are trimmed and still checked.
`*f=config.guess*` also matches `f=config.guess.backdoor`, so the exemption
still skipped verification on unrelated downloads. Second time a wildcard in
this exemption has been wider than intended — a domain glob before, a filename
prefix now — so drop wildcards entirely and match the two URLs as literals.

There is nothing left to widen: the definitions reference exactly these two
fixed strings, anything else is checked. Quoted so `?` and `;` are matched
literally rather than as glob and case-clause syntax. If the URLs ever change
shape, this list has to be updated by hand, which is the intended cost of
skipping verification on a download.

Verified the two real URLs still pass and four bypass shapes are caught:
config.guess.backdoor, config.subversion, an altered hb parameter, and the same
path on another host.
A backslash-newline is removed entirely in shell; it does not become
whitespace. Joining with a space split tokens that bash keeps together, so
`"https\` + `://host/pkg.tar.gz"` — one URL to bash, confirmed by sourcing it —
arrived at the extractor as `https ://host/pkg.tar.gz` and matched nothing.
Another download skipped silently.

Contrived to write by hand, but the joiner was approximating shell semantics
rather than following them, and that gap is what the check keeps getting caught
by. Ordinary continuations already carry whitespace around the backslash, so
nothing else changes.

Verified: real definitions pass, the split-scheme URL is now caught, and
continuations with digests still pass in all three forms.
Copilot AI review requested due to automatic review settings August 3, 2026 02:26

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ca195bf5ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 3 changed files in this pull request and generated no new comments.

Only full-line comments were stripped, so a reference link in a trailing comment
was treated as a download and failed for lacking a digest — blocking CI and
signoff over a URL that is never fetched. First false positive here rather than
a false negative, and the more disruptive direction: it stops work rather than
letting something through.

Stripping comments has to leave `pkg.tar.gz#<sha256>` alone, since cutting at
the first `#` would turn every checksummed URL into a failure. The `#` opening a
comment is always preceded by whitespace or starts the line; the `#` before a
digest never is, it follows the last character of the URL. Keying on that
separates them without parsing quotes.

Verified: a reference URL in a trailing comment is ignored, the digest on the
same line still registers, a full-line comment URL is ignored, and a genuinely
undigested download on a commented line is still caught.
Copilot AI review requested due to automatic review settings August 3, 2026 02:36

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 67ddaf4719

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bin/ci Outdated
`install_package ...;# note` is a comment to bash — verified, not assumed:
`bash -c 'echo one;# echo two'` prints only "one". The previous rule required
whitespace or line start before the `#`, so the trailing reference URL was read
as a download and failed the lint despite the real package URL being
checksummed. Same disruptive direction as the last one: it blocks CI over a URL
nothing fetches.

A `#` opens a comment when it starts a word, which includes straight after an
operator that terminated the previous one. Digests stay safe because a digest's
`#` never starts a word — it follows the last character of the URL, and no URL
character is in the operator set.

Verified against `;#`, `&&#` and a spaced `#`, all ignored, with the digests on
those same lines still registering and a genuinely undigested download after a
`;#` still caught.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants