Provision SCRAM users

Declare a Kafka SCRAM principal with a KafkaUser, and monedula-gitops creates its credential on the brokers.

Before you start

• Apache Kafka or Confluent Platform. KafkaUser does not work on Confluent Cloud (Supported platforms).
• A connecting principal with Describe and Alter on the Cluster resource, or a super-user.

Steps

1. Declare the user

The password source differs between the CLI and the operator, so the manifest does too. Save it as manifests/user.yaml:

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaUser
metadata:
  name: svc-orders-app
spec:
  clusterRef:
    name: shared
  username: svc-orders-app
  mechanism: SCRAM-SHA-512
  password:
    valueFrom:
      env: ORDERS_APP_PASSWORD

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaUser
metadata:
  name: svc-orders-app
spec:
  clusterRef:
    name: shared
  username: svc-orders-app
  mechanism: SCRAM-SHA-512
  password:
    generate: {}

• username defaults to metadata.name and cannot change once set.
• mechanism is SCRAM-SHA-256 or SCRAM-SHA-512, the default.
• iterations is optional, from 4096 to 16384. Without it, the broker default applies.
• password takes exactly one source. The CLI reads valueFrom.env or valueFrom.file, where a file path resolves relative to the directory of the cluster config file and not the manifest’s. The operator reads valueFrom.secretKeyRef or generates the password with generate: {}, which the CLI rejects when it loads the manifest. Both runtimes reject inline and configMapKeyRef for passwords.

With generate: {}, the operator generates the password from 32 random bytes, base64url-encoded, and stores it in a Secret named <KafkaUser name>-kafka-credentials, here svc-orders-app-kafka-credentials, with the keys username, password and mechanism and the label gitops.monedula.dev/credential-source: "true". It creates the Secret before it writes the credential to Kafka, so if the operator stops between the two, the next reconcile applies the stored password. The Secret has an owner reference to the KafkaUser, so Kubernetes garbage-collects it with the user. The operator’s RBAC has no delete on Secrets, and the operator never deletes it.

A Secret with that name that the KafkaUser does not own is never adopted or overwritten. The reconcile stops with ValidationFailed=True and Ready=False, both with the reason ForeignSecret and a message that names the Secret. Delete or rename that Secret, and the operator retries at the next resync.

To use a Secret you manage instead, point the operator at it:

password:
  valueFrom:
    secretKeyRef:            # operator mode
      name: svc-orders-app-credentials
      key: password

The Secret must be in the same namespace as the KafkaUser.

2. Grant the user access

A credential only lets the principal log in. Grant it access in the topic’s access block, saved as manifests/topic.yaml:

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaTopic
metadata:
  name: users-orders
spec:
  clusterRef:
    name: shared
  topicName: users.orders
  partitions: 1
  access:
    producers:
      - principal: "User:svc-orders-app"
        operations:
          - Write

3. Apply the manifests

export ORDERS_APP_PASSWORD=orders-app-secret
monedula-gitops apply -f ./manifests --cluster-config-file ./cluster.yaml

kubectl apply -f manifests/

The CLI reads the password at execute time, right before it writes the credential, and never shows it in plan output. In a pipeline, set the variable from your secrets manager.

4. Rotate the password

Kafka never exposes a SCRAM password, so monedula-gitops cannot detect a changed one as drift. You trigger a rotation explicitly:

• With the CLI, change the value behind the source, then apply with --rotate-passwords. It re-writes the password of every declared user that is otherwise in sync.

  monedula-gitops apply -f ./manifests --cluster-config-file ./cluster.yaml --rotate-passwords

• With the operator and secretKeyRef, update the Secret. The operator picks up the change at the next resync, or at once if the Secret has the label gitops.monedula.dev/credential-source: "true".

• With the operator and generate, delete the generated Secret svc-orders-app-kafka-credentials, and the operator generates a new password. Editing the KafkaUser never regenerates it.

5. Change the mechanism

Changing mechanism writes the credential under the new mechanism, then deletes the old one. If that delete fails, the operation fails and the user keeps both credentials. Drift detection compares only the declared mechanism, so diff, verify and the operator never report the old credential again, and neither another apply nor the next reconcile retries the delete. Remove it by hand, here the SCRAM-SHA-256 credential left by a change to SCRAM-SHA-512:

kafka-configs.sh --bootstrap-server <broker> --command-config admin.properties \
  --alter --delete-config 'SCRAM-SHA-256' --entity-type users --entity-name svc-orders-app

6. Remove a user

The CLI never deletes a user that is missing from your manifests.

Check it worked

monedula-gitops verify -f ./manifests --cluster-config-file ./cluster.yaml

kubectl get kafkauser svc-orders-app -o jsonpath='{.status.conditions}'
kubectl get kafkatopic users-orders -o jsonpath='{.status.conditions}'
kubectl get secret svc-orders-app-kafka-credentials

apply exits 0 and names svc-orders-app and users.orders in its output. verify compares the username, the mechanism and, when you set it, the iteration count, and exits 0 when they match. With the operator, the KafkaUser reports Ready and UserSynced as True; UserSynced is False, with the reason ApplyIncomplete and the failed operations in its message, when a SCRAM operation could not be applied. The KafkaTopic reports Ready and TopicAccessSynced as True, and the generated Secret exists. The broker now has a SCRAM-SHA-512 credential for svc-orders-app and a Write ACL on users.orders for User:svc-orders-app.

Related

• The KafkaUser reference lists every field.
• Grant producers and consumers access covers the access block in detail.
• Runnable version: scenarios/25-kafka-user