Git-simple to start · platform-grade to grow

Kollect

Kubernetes knows what's running right now. Kollect turns that into a durable record your whole platform can use — a Git history you can diff, a database your portal can query, an event stream your automation can react to. Declare what matters in a few CRs (select by GVK, extract with CEL), and every sink receives the same rows, in parallel.

Start with one Git repo. Grow to a whole platform. On day one, a single pipeline gives you a Git-committed inventory — git log is your audit trail, git diff is your drift report, no scripts, no apiserver hammering. As adoption grows, nothing gets rebuilt: the same rows fan out to Postgres, Kafka, and object storage, and KollectScope keeps it multi-tenant — every team owns its inventory as configuration, not code, in its own namespace. Consumers read export data, never unbounded list/watch against the live cluster.

Read the docs: platformrelay.github.io/Kollect — architecture, quick start, CR reference, ADRs, and examples. This README is the front door; the site is the map.

Pre-1.0. Kollect uses a v1alpha1 API. Breaking API or default changes may ship in minor releases before 1.0; release notes and migration guidance call them out. See the roadmap for current maturity.

Why Kollect?

• Decoupled read model — consumers query a sink, not the apiserver. No RBAC blast radius, no watch-storm risk, no etcd size limits (why).
• Event-driven, no polling — one shared informer per GVK keeps inventory current as the cluster changes (ADR-0301).
• Schema-flexible — declare the attributes you want in a KollectProfile; no bespoke collector per resource kind.
• Pluggable sinks, no privileged backend — the same snapshot fans out to Git, Postgres, object store, or an event stream (sink taxonomy).
• Multi-tenant by design — KollectScope gates which teams, namespaces, and sinks each tenant may use.
• Fleet-ready — N single-mode operators → one shared sink, partitioned by spec.cluster; no central hub tier to operate (ADR-0501).
• Scale-aware architecture — shared informers, export sharding, and tunable reconcile/dispatch concurrency; the performance guide separates measured evidence from targets (performance).

See it end-to-end

A real pipeline is a handful of Kubernetes resources. The first-inventory walkthrough collects container images from Deployments and exports them to Git for an inspectable audit trail:

flowchart LR
  Profile["<b>KollectProfile</b><br/>Deployment schema"]
  Target["<b>KollectTarget</b><br/>select Deployments"]
  Inv["<b>KollectInventory</b><br/>aggregate · debounce · export"]
  Snap["<b>KollectSnapshotSink</b>"]
  Db["<b>KollectDatabaseSink</b>"]
  Ev["<b>KollectEventSink</b>"]
  K8s[("Kubernetes API")]

  Profile --> Target
  K8s -- "informer per GVK" --> Target
  Target --> Inv
  Inv --> Snap
  Inv --> Db
  Inv --> Ev
  Snap --> SnapOut["Git · GitLab · S3 · GCS"]
  Db --> DbOut["Postgres · MongoDB"]
  Ev --> EvOut["Kafka"]

Loading

Quick start (MVP)

Spin up the full pipeline on a local kind cluster in one command (needs Docker, kind, kubectl, and Task):

git clone https://github.com/platformrelay/kollect.git && cd kollect
task dev-up                       # build, create kind cluster, install operator + sample CRs
kubectl get kinv,ktgt,ksnap,kdb -A    # watch the pipeline come up

task dev-up builds the manager, boots a kollect-dev kind cluster, installs the operator, and applies the sample Profile → Sink → Target → Inventory pipeline. Watch the KollectInventory Ready condition, then read your sink — the live demo repo shows what the Git export looks like.

Full walkthrough — local evaluation and Helm install: Install Kollect →

How it works

flowchart LR
  API["Kubernetes API"] -->|shared informers| Snapshot["Canonical inventory snapshot"]
  Snapshot -->|debounce| Inventory["KollectInventory"]
  Inventory --> SnapshotSinks["Git · GitLab · S3 · GCS"]
  Inventory --> DatabaseSinks["Postgres · MongoDB · BigQuery"]
  Inventory --> EventSinks["Kafka · NATS"]

Loading

The in-memory snapshot per inventory is canonical; every sink is a projection of it — no single backend is privileged (sink roles). Sinks are split into three CRD families (ADR-0414):

Sink family	Examples	Good for
KollectSnapshotSink	Git, GitLab, S3, GCS	Audit, diff, GitOps-friendly history
KollectDatabaseSink	Postgres, MongoDB	Rich queries for portals and dashboards
KollectEventSink	Kafka, NATS	Change streams, downstream consumers

Supported & planned sinks

Honest maturity tiers — see the roadmap for release timing.

Family CRD	spec.type	Status
KollectSnapshotSink	git	Core — production-ready
KollectSnapshotSink	gitlab	Core
KollectSnapshotSink	s3	Core
KollectSnapshotSink	gcs	Beta — shipped, maturing
KollectDatabaseSink	postgres	Core
KollectDatabaseSink	mongodb	Beta
KollectDatabaseSink	bigquery	Beta — analytics SQL
KollectEventSink	kafka	Beta
KollectEventSink	nats	Beta — JetStream emitter
KollectSnapshotSink	azureblob	Planned — needs real backend (roadmap)
KollectSnapshotSink	s3, gcs with serialization.format: parquet	Beta — shipped object-store output mode

Full payload lives in sinks; CR .status holds summaries only (etcd limits).

Performance

Kollect is designed for large single clusters and multi-cluster fleets. The performance guide distinguishes reproducible results from design targets and documents tuning for reconcile concurrency, export debounce, and sharding. Fleet fan-in uses shared sinks rather than a hub merge tier.

Learn more

Topic	Link
Problem statement, CRD model, reconciliation	Architecture
Locked platform decisions	Platform decisions
CR fields, RBAC, failure modes	CR reference
Multi-cluster fleet	ADR-0501
Sink taxonomy (state vs stream)	ADR-0401
Shipped, next, and later work	Roadmap
Examples index	Examples
Example: Deployment → Git export	Walkthrough
Live demo inventory (Git sink)	kollect-inventory-demo

Developers: run task lint, task test, and task verify before opening a PR — CONTRIBUTING.md.

Community

Contributing	CONTRIBUTING.md — DCO, PR workflow, good first tasks
Code of Conduct	CODE_OF_CONDUCT.md — Contributor Covenant v2.1
Governance	GOVERNANCE.md — roles, decisions, continuity

Security

Report vulnerabilities privately — see SECURITY.md. Security architecture: docs/ASSURANCE-CASE.md.

License

Copyright (c) 2026 Konrad Heimel. Licensed under the MIT License.