Replace deprecated TopicPolicy with ResourcePolicy, adopt template layout - #8
Open
chuck-alt-delete wants to merge 10 commits into
Open
chuck-alt-delete wants to merge 10 commits into
chuck-alt-delete wants to merge 10 commits into
Conversation
…yout TopicPolicy is deprecated in favour of ResourcePolicy, and this repo had drifted from the structure we recommend for real rollouts. Restructure it to mirror conduktor/self-service-template, minus the CI/CD wiring that a quick start doesn't need. - Replace central-team-repo/ and application-team-repo/ with platform/ and applications/<app>/<instance>/, matching the template. - Replace the four TopicPolicy files with the template's seven ResourcePolicies, and switch ApplicationInstance topicPolicyRef -> policyRef. Update apiVersion self-service/v1 -> self-serve/v1. - Swap the Redpanda compose stack for the three-broker Kafka stack from cs-playground, so topic-rules-prod (RF 3, ISR 2) ships unmodified. Adds Schema Registry so subject-rules has something to validate, and Gateway for experimenting afterwards. - Declare the Kafka cluster as code in platform/clusters/ rather than via CDK_CLUSTERS_* env vars. - Drop the unused clickstream and wikipedia applications. - Add platform/exceptions/ with a worked example: AdminToken bypasses ResourcePolicy validation, so a platform-owned directory is how an approved exception gets granted. - Require CDK_LICENSE. The whole Self-service API is wrapped in disabledInFreeLicense, so the previous licence-free compose could not have worked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The setup flow made you log into the UI, copy an API key out of Settings, export it on the host and recreate the CLI container — and did the same dance again to switch tokens. `token create admin` exists precisely to mint a first token from CDK_USER/CDK_PASSWORD, so use it. - Give conduktor-ctl CDK_USER/CDK_PASSWORD instead of CDK_API_KEY. In the default auth mode `apply` only ever reads CDK_API_KEY, so the two coexist without ambiguity. - Add a /api/health/live healthcheck to Console and make the CLI depend on it, so `up --wait` returns when Console is actually ready rather than merely running. - Token switching now happens inside the container with an export, with no restarts at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Setting CDK_USER/CDK_PASSWORD on the conduktor-ctl service put admin credentials in a file that gets committed, and put them alongside CDK_API_KEY. That combination is not merely redundant: with CDK_AUTH_MODE=external the CLI exits with "Can't set both CDK_API_KEY and CDK_USER" (pkg/client/console_client.go), so anyone adapting this stack to SSO would hit a hard failure. conduktor-ctl now carries only CDK_BASE_URL. Credentials go to the single `token create admin` invocation via `docker compose exec -e`, and the resulting token is injected the same way when opening the shell. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both surfaced against Console 1.47.1 on a clean stack.
ApplicationInstance spec.policyRef rejects ApplicationGroup policies:
allowedApplicationInstancePolicies is {Connector, Topic, Subject}, so
listing appgroup-restrictions there failed both instances with "Policy
with name 'appgroup-restrictions' has ApplicationGroup but only
[Connector, Topic, Subject] are allowed". Application spec.policyRef has
no such restriction, so it moves there — verified enforcing from that
level, rejecting both direct members and prod write permissions.
metadata.descriptionIsEditable must be a real boolean. The quoted "true"
carried over from the old repo is rejected with "Got value '\"true\"'
with wrong type, expecting 'true' or 'false'".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The README pointed at docs.conduktor.io for every actual step, so the repo didn't stand on its own. It now carries the full walkthrough: start the stack, mint a token, apply the platform resources, apply the team's resources, watch a policy reject a topic, grant an exception, tear down. Commands pass their token per invocation with `docker compose exec -e CDK_API_KEY=$ADMIN_TOKEN` rather than opening an interactive shell and exporting inside it. Each step is now copy-pasteable on its own and it's visible at a glance which identity a command runs as — $ADMIN_TOKEN or $APP_TOKEN. Also adds a healthcheck to Postgres and makes Console wait for it. With `condition: service_started` Console could start before Postgres finished initdb, exhaust its connection retries and exit 1, taking the whole stack down. Postgres 18 initializes slowly enough to hit this reliably. Verified by extracting every bash block from the README's tutorial section and running them as a script against a clean stack: all steps apply, and the deliberate violation is rejected with the documented message. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
chuck-alt-delete
marked this pull request as ready for review
September 16, 2026 20:44
Mirrors the tutorial rewrite in conduktor-docs so the two don't diverge. Drawn from the self_service_resource_mgmt and data_ownership_governance narratives in the ROI calculator: thorough review makes the central team the bottleneck, fast review lets misconfigurations through, and access requests are the harder half because the central team approves them without knowing data sensitivity. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Mirrors the tutorial change in conduktor-docs. The record Self-service builds — which application owns which topics and service accounts — is what lineage, chargeback, alerts and the Topic Catalog run on, verified against console-plus rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TopicPolicyis deprecated in favour ofResourcePolicy, and this repo had drifted from the structure we recommend for real rollouts. This restructures it to mirror conduktor/self-service-template, minus the CI/CD wiring a quick start doesn't need.The tutorial rewrite in conduktor-docs#1550 is a companion PR. Both should merge together.
Structure
central-team-repo/+application-team-repo/platform/+applications/<app>/<instance>/TopicPolicyfilesResourcePolicyfiles, ported from the templatetopicPolicyRefpolicyRefapiVersion: self-service/v1self-serve/v1CDK_CLUSTERS_*env varsplatform/clusters/, applied with the CLIThe compose stack comes from the
local-kafka-gateway-passthrough-console-minimalpov-quickstart. Three brokers meanstopic-rules-prodships unmodified —replicationFactor: 3andmin.insync.replicas: 2actually work, so the dev-versus-prod contrast is real rather than annotated. Schema Registry was added because the stack had none andsubject-ruleswould have had nothing to validate.platform/exceptions/gets a worked example. AnAdminTokenbypassesResourcePolicyvalidation, so "the platform team applies it with an admin token" is literally the mechanism, not a convention.Setup is shorter
conduktor token create adminmints the first token fromCDK_USER/CDK_PASSWORD, so there's no key to copy out of the UI. Credentials are passed to that one command viadocker compose exec -erather than being set on the service —CDK_USERalongsideCDK_API_KEYis rejected outright underCDK_AUTH_MODE=external. Console also gained an/api/health/livehealthcheck, soup --waitreturns when Console is ready rather than merely running.Requires a license
The whole Self-service API is wrapped in
disabledInFreeLicense, so every call 403s on the Free plan. The previous compose set no license at all and could not have worked end to end.CDK_LICENSEis now required andstart.shfails fast without it.Verification
Ran the full tutorial twice against Console 1.47.1, the second time on a torn-down-and-rebuilt stack. Every step applies with zero errors: 2 clusters, 3 groups, 7 policies, 1 application + 2 instances, 8 topics, 2 subjects, 2 application groups. The deliberate violation produces the documented error, all three suggested variations behave as described, and the exception applies with the admin token.
Two errors were found and fixed that way:
ApplicationInstance.spec.policyRefrejectsApplicationGrouppolicies, soappgroup-restrictionsmoved to the Applicationmetadata.descriptionIsEditablemust be a real boolean; the quoted"true"carried over from the old repo is rejectedSince this PR was opened
docker compose exec -e CDK_API_KEY=$ADMIN_TOKEN) instead of opening an interactive shell and exporting inside it. Each step is copy-pasteable on its own, and it's visible at a glance which identity a command runs as.condition: service_started, Console could start before Postgres finishedinitdb, exhaust its connection retries and exit 1, taking the whole stack down. Postgres 18 initializes slowly enough to hit this reliably. Postgres now has apg_isreadyhealthcheck and Console waits for it.The README was re-verified by extracting every bash block from its tutorial section and running them as a script against a clean stack.
🤖 Generated with Claude Code