A dedicated Kafka Connect image to be leveraged with Streams for Apache Kafka
Currently the Connect image only contains the Debezium connector for PostgreSQL, any other connectors or plugins required would need to be added in the future.
Plugins are downloaded directly from Maven Central during the image build and verified against SHA-256 digests pinned in the Dockerfile. To add connectors or upgrade versions, update DEBEZIUM_MAVEN_VERSION and the corresponding digests in the Dockerfile.
Here are the current list of Kafka Connectors managed by the Fabric Kessel team which are deployed via this repo
| Connector Name | Deploy File | Service Provider | Purpose |
|---|---|---|---|
| kessel-inventory-api-connector | kafkaconnect-w-auth.yml | Kessel | Used to capture all outbox events from the Inventory API DB outbox table for Relations replication |
| hbi-migration-connector | hbi-hosts-migration-connector.yml | HBI | Used to perform an initial migration from HBI's hosts table |
| hbi-outbox-connector | hbi-outbox-connector.yml | HBI | Used to capture all outbox events from HBI's DB outbox table for replication |
Log in to the required registries, then run the docker-build-push target with your image destination. The base image is pulled from registry.redhat.io during the build, so both logins are required.
podman login quay.io # or: docker login quay.io
podman login registry.redhat.io # or: docker login registry.redhat.io
make docker-build-push IMAGE=quay.io/your-org/kessel-kafka-connectIf the build fails with an authentication error mentioning registry.redhat.io, ensure you are logged in using your Red Hat Customer Portal credentials. See the Red Hat Registry Authentication guide for details.
podman is used automatically if available, otherwise docker is used. Override with DOCKER=docker make docker-build-push IMAGE=quay.io/your-org/kessel-kafka-connect.
The KafkaConnect CR templates can be used to deploy a Kafka Connect cluster using the image built with the provided Dockerfile. There are 2 versions of the KafkaConnect CR: one with authentication, one without.
The kafkaconnect-ephem.yml template is used for Ephemeral testing and can be deployed via bonfire. Note, the Clowder-provided Kafka cluster is used for the Connect cluster and does not require any authentication. The deployment also includes the Kessel Inventory outbox connector, and HBI outbox connector to simplify test deployments for service providers as well as the Mgmt Fabric teams.
bonfire deploy kessel -C kessel-kafka-connect
# Kessel Inventory also lists KKC as a dependency and can be deployed automatically while deploying Inventory API
bonfire deploy kessel -C kessel-inventoryThe kafkaconnect-w-auth.yml template is used for Stage/Prod and relies on AWS MSK. It is configured with SASL/SCRAM and requires credentials to authenticate to the cluster. In order to authenticate, you will need a Kakfa user configured for the MSK cluster. See the Managed Streaming for Apache Kafka (MSK) via App-Interface section of the App Interface docs on how to add users.
Log levels for Kafka Connect and Debezium can be configured independently using template parameters:
| Parameter | Default | Description |
|---|---|---|
CONNECT_LOG_LEVEL |
INFO |
Log level for Kafka Connect |
DEBEZIUM_LOG_LEVEL |
INFO |
Log level for Debezium connectors |
Valid values: INFO, DEBUG, TRACE
Logs include Debezium MDC and Kafka Connect context for easier filtering:
2026-02-05 15:00:18,237 INFO [Postgres|streaming] [kessel-inventory-source-connector|task-0] Processing messages [io.debezium...]
Format: [connectorType|activity] [connectorName|taskId] message [loggerClass]
Filtering examples:
# Filter by connector type
oc logs <pod> | grep "\[Postgres|"
# Filter by connector name
oc logs <pod> | grep "kessel-inventory-source-connector"
# Filter by activity (snapshot, streaming)
oc logs <pod> | grep "|streaming\]"
oc logs <pod> | grep "|snapshot\]"