Merge #53: port the T4 table as an automated test against the test bu… #81

Workflow file for this run

	name: CI

	on:
	push:
	branches: [main]
	pull_request:
	branches: [main]
	workflow_dispatch:

	permissions:
	contents: read

	# Two mechanisms, one per event kind:
	# - PR runs share a group per branch (`github.head_ref` is non-empty only on
	# `pull_request`), and `cancel-in-progress` cancels the superseded one.
	# - Everything else (`push`, `workflow_dispatch`) is keyed on `github.sha`,
	# unique per commit, so each `main` run sits alone in its own group.
	# The second is not redundant with the first. A group holds at most one
	# pending run regardless of `cancel-in-progress`, so under one branch-keyed
	# group a third push evicts the second's queued run before it ever starts —
	# a commit lands on the protected branch with no CI signal at all (#44).
	concurrency:
	group: ${{ github.workflow }}-${{ github.head_ref \|\| github.sha }}
	cancel-in-progress: ${{ github.event_name == 'pull_request' }}

	jobs:
	lint:
	name: lint
	runs-on: ubuntu-latest
	# WIF: mint an OIDC token for google-github-actions/auth to impersonate the
	# read-only co-pypi-reader SA and pull the cannobserv wheelhouse.
	permissions:
	contents: read
	id-token: write
	steps:
	- uses: actions/checkout@v5

	- name: Install uv
	uses: astral-sh/setup-uv@v5
	with:
	enable-cache: true

	- name: Install Python
	run: uv python install 3.12

	# co-core / co-core-aio resolve from a wheelhouse mirrored from the
	# private GCS index. Authenticate keyless via Workload Identity Federation
	# — the read-scoped provider impersonating the objectViewer-only
	# co-pypi-reader SA — then sync before `uv sync`.
	# Fail fast and legibly when the org variable is not visible to this repo.
	# google-github-actions/auth validates its own inputs before contacting
	# Google, so an empty provider surfaces as "must specify exactly one of
	# workload_identity_provider or credentials_json" — which reads like a
	# workflow-authoring bug and sends you looking at GCP IAM, where the
	# problem is not. Assert the precondition here instead.
	- name: Assert wheelhouse auth is configured
	env:
	WIF_PROVIDER: ${{ vars.GCP_WIF_PROVIDER }}
	run: \|
	if [ -z "$WIF_PROVIDER" ]; then
	echo "::error::vars.GCP_WIF_PROVIDER is empty for this repo. GCP is not the problem — the org variable is not visible here. Fix under Org Settings > Secrets and variables > Actions > Variables > GCP_WIF_PROVIDER > Repository access."
	exit 1
	fi
	echo "WIF provider: $WIF_PROVIDER"

	- name: Authenticate to Google Cloud (WIF, read-only)
	uses: google-github-actions/auth@v2
	with:
	workload_identity_provider: ${{ vars.GCP_WIF_PROVIDER }}
	service_account: co-pypi-reader@co-gcs.iam.gserviceaccount.com

	- name: Sync cannobserv wheelhouse
	run: uv run --no-project --with 'google-cloud-storage>=2,<4' python scripts/sync_wheelhouse.py

	- name: uv sync
	run: uv sync --frozen --group dev

	- name: ruff check
	run: uv run ruff check .

	- name: ruff format --check
	run: uv run ruff format --check .

	test:
	name: test
	runs-on: ubuntu-latest
	# WIF: mint an OIDC token for google-github-actions/auth (see the lint job).
	permissions:
	contents: read
	id-token: write
	steps:
	# `submodules` is load-bearing for tests/test_skills_hook.py, which
	# dereferences the .claude/hooks/ symlink into skills-vendor/ — an
	# unpopulated submodule makes that link dangle and the test cannot pass.
	# Both submodules are public HTTPS in .gitmodules, so no token is needed.
	# The coupling this buys: an upstream force-push that GCs a pinned SHA
	# fails the job at checkout, which looks nothing like a test failure.
	# The lint job deliberately omits this — ruff extend-excludes
	# skills-vendor/ (pyproject.toml), so it never reads what it does not
	# fetch. Costs ~2.2s of a ~1m10s job. See docs/SKILLS.md.
	- uses: actions/checkout@v5
	with:
	submodules: true

	- name: Install uv
	uses: astral-sh/setup-uv@v5
	with:
	enable-cache: true

	- name: Install Python
	run: uv python install 3.12

	# Fail fast and legibly when the org variable is not visible to this repo.
	# google-github-actions/auth validates its own inputs before contacting
	# Google, so an empty provider surfaces as "must specify exactly one of
	# workload_identity_provider or credentials_json" — which reads like a
	# workflow-authoring bug and sends you looking at GCP IAM, where the
	# problem is not. Assert the precondition here instead.
	- name: Assert wheelhouse auth is configured
	env:
	WIF_PROVIDER: ${{ vars.GCP_WIF_PROVIDER }}
	run: \|
	if [ -z "$WIF_PROVIDER" ]; then
	echo "::error::vars.GCP_WIF_PROVIDER is empty for this repo. GCP is not the problem — the org variable is not visible here. Fix under Org Settings > Secrets and variables > Actions > Variables > GCP_WIF_PROVIDER > Repository access."
	exit 1
	fi
	echo "WIF provider: $WIF_PROVIDER"

	- name: Authenticate to Google Cloud (WIF, read-only)
	uses: google-github-actions/auth@v2
	with:
	workload_identity_provider: ${{ vars.GCP_WIF_PROVIDER }}
	service_account: co-pypi-reader@co-gcs.iam.gserviceaccount.com

	- name: Sync cannobserv wheelhouse
	run: uv run --no-project --with 'google-cloud-storage>=2,<4' python scripts/sync_wheelhouse.py

	- name: uv sync
	run: uv sync --frozen --group dev

	# Validates the wheelhouse end to end: the extras the MVP loop depends on
	# ([extract] for fingerprinting, [bus] for the Redis Streams driver) must
	# import, not merely resolve.
	- name: co-core import smoke (extras wired)
	run: \|
	uv run python -c "
	from co_core.pure.util.hashing import sha256
	from co_core.pure.extract import simhash
	from co_core.pure.adapters.bus import streams
	from co_core.pure.models.changes import BlobAvailableEvent, ContentFetchCommand
	from co_core_aio.bus import AsyncBusConsumer, AsyncBusPublisher
	from co_core_aio.fetch import AsyncFetchDriver
	assert streams.CONTENT_FETCH == 'content.fetch'
	assert streams.CONTENT_BLOBS == 'content.blobs'
	print('co-core extras OK')
	"

	# No Postgres service and no alembic steps: Replicator is DB-free. Its
	# durable state is the Redis consumer group's PEL plus content-addressed
	# blobs on disk. Unit tests use fakeredis; anything needing a real broker
	# is marked @pytest.mark.integration and excluded by default.
	- name: pytest (default, excludes integration)
	run: uv run pytest

	# The only job in this workflow that holds a delete-capable identity, and it is
	# a separate job for exactly that reason (#53). The `test` job authenticates as
	# `co-pypi-reader` — read on the package index and nothing else — and giving it
	# a second identity that can write and delete objects would widen every unit
	# test's blast radius to buy coverage that three tests need.
	#
	# Keyless: the test SA is bound to this repo's WIF provider, so no key exists to
	# leak. `google-github-actions/auth` writes a credentials file and exports
	# GOOGLE_APPLICATION_CREDENTIALS; the fixtures in tests/conftest.py strip that
	# variable from every test and re-point it, per test, from
	# REPLICATOR_TEST_GCS_CREDENTIALS — so the path is threaded explicitly rather
	# than inherited, and an unmarked test cannot reach the identity at all.
	gcs:
	name: gcs (T4 against the test bucket)
	runs-on: ubuntu-latest
	permissions:
	contents: read
	id-token: write
	steps:
	- uses: actions/checkout@v5

	- name: Install uv
	uses: astral-sh/setup-uv@v5
	with:
	enable-cache: true

	- name: Install Python
	run: uv python install 3.12

	- name: Assert wheelhouse auth is configured
	env:
	WIF_PROVIDER: ${{ vars.GCP_WIF_PROVIDER }}
	run: \|
	if [ -z "$WIF_PROVIDER" ]; then
	echo "::error::vars.GCP_WIF_PROVIDER is empty for this repo. GCP is not the problem — the org variable is not visible here. Fix under Org Settings > Secrets and variables > Actions > Variables > GCP_WIF_PROVIDER > Repository access."
	exit 1
	fi

	# First identity: read-only, and only to mirror the wheelhouse.
	- name: Authenticate to Google Cloud (WIF, read-only)
	uses: google-github-actions/auth@v2
	with:
	workload_identity_provider: ${{ vars.GCP_WIF_PROVIDER }}
	service_account: co-pypi-reader@co-gcs.iam.gserviceaccount.com

	- name: Sync cannobserv wheelhouse
	run: uv run --no-project --with 'google-cloud-storage>=2,<4' python scripts/sync_wheelhouse.py

	- name: uv sync
	run: uv sync --frozen --group dev

	# Second identity, after the wheelhouse is already mirrored: objectAdmin on
	# co-gcs-test-replication and no grant whatsoever on the production bucket
	# (docs/DEPLOYMENT.md, "The GCS test bucket").
	- name: Authenticate to Google Cloud (WIF, test bucket writer)
	id: gcs_auth
	uses: google-github-actions/auth@v2
	with:
	workload_identity_provider: ${{ vars.GCP_WIF_PROVIDER }}
	service_account: co-gcs-test-replicator@co-gcs.iam.gserviceaccount.com

	# Skipping is not passing here, and a skip is silent — the fixtures skip a
	# `gcs` test when either variable is unset, which on a developer's machine
	# is correct and in CI would mean the provisioning or the WIF binding
	# regressed while the job stayed green. The bucket is a literal below, so
	# only the credentials path can be empty; assert it rather than discover it.
	- name: Assert the test identity resolved
	run: \|
	if [ -z "${{ steps.gcs_auth.outputs.credentials_file_path }}" ]; then
	echo "::error::WIF auth produced no credentials file for co-gcs-test-replicator. The gcs tests would skip, not fail. Check the SA's roles/iam.workloadIdentityUser binding on this repo's principalSet (docs/DEPLOYMENT.md)."
	exit 1
	fi

	# --no-cov: these rows do not exercise all of src/, and the gate measures
	# it. `-m gcs` is deselected from the default suite, so this is the only
	# place they run. `-rs` prints the reason for any skip that survives the
	# assertion above, so a future one is legible in the log rather than a
	# missing line.
	- name: pytest (T4 rows against the real bucket)
	env:
	REPLICATOR_TEST_GCS_BUCKET: co-gcs-test-replication
	REPLICATOR_TEST_GCS_CREDENTIALS: ${{ steps.gcs_auth.outputs.credentials_file_path }}
	run: uv run pytest --no-cov -m gcs -rs

Navigation Menu

Uh oh!

Merge #53: port the T4 table as an automated test against the test bu… #81

Workflow file

Merge #53: port the T4 table as an automated test against the test bu… #81

Uh oh!

Workflow file for this run