monedula-gitops docs menu
Get started with the operator
Run the monedula-gitops operator on a local k3d cluster with Helm and let it reconcile a KafkaTopic and its access from a custom resource.
On this page
Everything runs inside a local k3d cluster: Kafka, Schema Registry, cert-manager and an operator image built from the cloned source.
What you need
- Docker, to run the cluster and build the operator image.
- k3d, which runs a k3s Kubernetes cluster in Docker.
- kubectl.
- Helm 3.
- Git and a bash shell. The commands on this page use bash syntax.
1. Get the quickstart files
Clone the repository and change into the Kubernetes quickstart directory:
git clone https://github.com/monedula-dev/monedula-gitops.git
cd monedula-gitops/quickstart/k8s
Run every command in this tutorial from this directory.
2. Run the setup script
./setup.sh
In order, the script:
- Creates the k3d cluster
monedula, or reuses it. - Builds the operator image with
docker buildand imports it withk3d image import. - Installs cert-manager v1.16.3, which issues the admission webhook’s serving certificate.
- Deploys a single-node Kafka and a Schema Registry into the
kafkanamespace, and waits for the Job that creates the SCRAM users. - Installs the chart into the
monedula-systemnamespace withhelm upgrade --install, the local image and--set webhook.enabled=true. - Applies the
monedula-demonamespace, a Secret namedkafka-credentialsand three sample resources.
When it finishes, it prints the commands to run next.
3. Read the resources the script applied
The sample KafkaCluster, operator/samples/kafkacluster.yaml, reads the Kafka SASL/SCRAM and Schema Registry credentials from the kafka-credentials Secret with secretKeyRef:
apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaCluster
metadata:
name: demo
namespace: monedula-demo
spec:
bootstrapServers: kafka.kafka.svc.cluster.local:9092
auth:
mechanism: SCRAM-SHA-512
scram:
username:
valueFrom:
secretKeyRef:
name: kafka-credentials
key: kafka-username
password:
valueFrom:
secretKeyRef:
name: kafka-credentials
key: kafka-password
schemaRegistry:
endpoint: http://schema-registry.kafka.svc.cluster.local:8081
auth:
type: basic
username:
valueFrom:
secretKeyRef:
name: kafka-credentials
key: sr-username
password:
valueFrom:
secretKeyRef:
name: kafka-credentials
key: sr-password
The operator reads Secrets from the resource’s own namespace, so the Secret and all the sample resources live in monedula-demo. The Secret holds the SCRAM super-user admin, which bypasses ACL enforcement; Connect to a secured Kafka cluster explains how a principal without super-user rights can lock itself out.
The sample KafkaTopic, operator/samples/kafkatopic.yaml, declares the topic payments.orders and its access in one resource:
apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaTopic
metadata:
name: orders
namespace: monedula-demo
spec:
clusterRef:
name: demo
topicName: payments.orders
partitions: 3
config:
retention.ms: "604800000"
access:
producers:
- principal: User:orders-service
operations:
- Write
consumers:
- principal: User:billing-service
group: billing-consumers
topicOperations:
- Read
groupOperations:
- Read
reconciliation:
mode: Enforce
deletionPolicy: Orphan
User:orders-service may write to the topic, and User:billing-service may read it with the consumer group billing-consumers. deletionPolicy: Orphan keeps the Kafka topic if you delete the resource. The topic has no schema because in operator mode a schema body cannot come from a local file.
The third sample, a KafkaAccessPolicy named billing-access, grants User:billing-service Read and Describe on every topic whose name starts with the prefix payments., as a prefixed ACL.
4. Check that the operator is running
kubectl -n monedula-system get pods
The operator pod should be Running and ready.
5. Watch the topic reconcile
List the sample resources:
kubectl -n monedula-demo get kafkaclusters,kafkatopics,kafkaaccesspolicies
Inspect the topic and print its phase:
kubectl -n monedula-demo describe kafkatopic orders
kubectl -n monedula-demo get kafkatopic orders -o jsonpath='{.status.phase}{"\n"}'
When reconciliation has finished:
KafkaCluster/demoreports a healthy connection: the operator reached the broker and Schema Registry with the credentials it read from the Secret.KafkaTopic/ordersreaches the phaseReadywith the conditionTopicSynced=True, and the Kafka topicpayments.ordersexists with 3 partitions.KafkaAccessPolicy/billing-accesshas reconciled its prefixed topic ACL.
If a resource stays in another phase, kubectl describe shows the most recent error, and the operator logs show what it is doing:
kubectl -n monedula-system logs deploy/monedula-gitops -f
6. See the admission webhook reject a conflict
Try to create a second KafkaTopic that claims the same topic name on the same cluster as orders:
kubectl -n monedula-demo apply -f - <<EOF
apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaTopic
metadata:
name: orders-dup
namespace: monedula-demo
spec:
clusterRef:
name: demo
topicName: payments.orders
partitions: 3
deletionPolicy: Orphan
EOF
The admission webhook rejects the request:
Error from server (Forbidden): error when creating "STDIN":
admission webhook "vkafkatopic.gitops.monedula.dev" denied the request:
KafkaTopic monedula-demo/orders-dup conflicts with monedula-demo/orders:
both resolve to topicName "payments.orders" on cluster "demo" ...
A topic’s name cannot change after creation either, because a rename would orphan the old Kafka topic. Try to rename orders:
kubectl -n monedula-demo patch kafkatopic orders --type merge \
-p '{"spec":{"topicName":"payments.orders-renamed"}}'
# Error: spec.topicName is immutable: resolved topic name cannot change ...
7. Delete the topic from Kafka
Under deletionPolicy: Orphan, deleting the KafkaTopic resource would leave the Kafka topic in place. To delete the topic from Kafka, switch the policy to Delete and approve the deletion with the gitops.monedula.dev/allow-delete annotation.
kubectl -n monedula-demo patch kafkatopic orders --type merge \
-p '{"spec":{"deletionPolicy":"Delete"}}'
kubectl -n monedula-demo annotate kafkatopic orders \
gitops.monedula.dev/allow-delete=true --overwrite
kubectl -n monedula-demo delete kafkatopic orders
Without the annotation, the operator would keep the topic and its ACLs, even with deletionPolicy: Delete; Delete topics from Kubernetes shows what the waiting resource reports then.
8. Tear down
./teardown.sh
teardown.sh uninstalls the operator’s Helm release and deletes the k3d cluster monedula, with everything this tutorial created.
Next steps
- One model, two runtimes explains what the operator shares with the CLI and where they differ.
- Configure the Kubernetes operator covers the Helm values for production installs.
- Enforce tenancy with admission webhooks restricts which topics each namespace may manage.