Topic-centric access

The principals that produce to and consume from a topic are declared on the KafkaTopic itself, in its access block, and monedula-gitops compiles them into the ACLs the broker needs. A topic and the applications that use it are one manifest.

Why access lives on the topic

Most Kafka tooling keeps a topic and its access in separate objects, and keeping two objects in sync is where drift creeps in; How monedula-gitops compares lists where each tool declares access. Here the everyday case stays in one place:

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaTopic
metadata:
  name: orders
spec:
  clusterRef: { name: prod }
  topicName: orders
  partitions: 6
  config:
    retention.ms: "604800000"
  access:
    producers:
      - principal: User:svc-checkout
    consumers:
      - principal: User:svc-fraud
        group: fraud-orders

A reviewer sees the topic and everyone who can use it in the same diff, and deleting a KafkaTopic under deletionPolicy: Delete, with the allow-delete annotation, deletes its topic-local ACLs too.

What the access block compiles to

Every entry compiles to Allow ACLs with a literal pattern on the topic’s topicName:

• access.producers[] needs a principal. It compiles to topic ACLs for its operations, which default to [Write, Describe].
• access.consumers[] needs a principal and a group. It compiles to topic ACLs for its topicOperations, which default to [Read, Describe], and to group ACLs on group for its groupOperations, which default to [Read].

For the manifest above, User:svc-checkout gets Write and Describe on the topic orders, and User:svc-fraud gets Read and Describe on the topic plus Read on the consumer group fraud-orders. An entry’s optional host, * by default, restricts its ACLs to one source host or CIDR. A host of only whitespace is a validation error; leave the field out for all hosts.

On Confluent Platform clusters that authorize with RBAC, the same block can compile to MDS role bindings instead of, or as well as, ACLs; Manage Confluent RBAC role bindings shows the mapping.

When to use KafkaAccessPolicy instead

Some access is not owned by exactly one topic. A KafkaAccessPolicy holds raw Kafka ACL rules for those cases:

• prefixed patterns, such as every topic whose name starts with billing.
• rules on cluster, transactionalId or delegationToken resources
• Deny rules
• host-restricted rules
• shared consumer groups

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaAccessPolicy
metadata:
  name: billing-access
spec:
  clusterRef:
    name: prod-eu
  rules:
    - principal: User:svc-billing
      host: "10.0.0.15"
      resource:
        type: topic
        name: billing.
        patternType: prefixed
      operations: [Read, Write, Describe]
  deletionPolicy: Orphan

For Confluent RBAC roles beyond producer and consumer, such as ResourceOwner or cluster-scoped roles, use a KafkaRoleBinding.

The principal’s credential

A KafkaUser declares the SCRAM credential behind a principal, so the topic, its access and the principal that uses it can all live in Git:

apiVersion: gitops.monedula.dev/v1alpha1
kind: KafkaUser
metadata:
  name: svc-checkout
spec:
  clusterRef: { name: prod }
  password:
    valueFrom: { env: SVC_CHECKOUT_PASSWORD }

How topic access and policies fit together

Topic-local access and access policies feed the same set of desired ACLs for a cluster. The same principal, host, resource and operation granted Allow by one and Deny by the other is a conflict. Ownership is per principal and resource pattern across both: once any manifest declares a pair, every live ACL for it is in the managed scope, so keep each pair declared in exactly one manifest set.

import cluster reconstructs topic-local producer and consumer access by folding ACLs that match a topic exactly onto its KafkaTopic. Prefixed or ambiguous ACLs become KafkaAccessPolicy rules.

Related

• Grant producers and consumers access is the step-by-step recipe.
• The KafkaTopic reference and KafkaAccessPolicy reference list every field.