Monedula
monedula-gitops docs menu

Import an existing Kafka cluster

Generate monedula-gitops manifests from a live cluster with import cluster, including schemas, and verify the round trip reports no drift.

How-to guides 15 of 18

5 min read

Updated 9 October 2026

Markdown

On this page

import cluster reads the live state of a Kafka cluster and writes manifests that describe it; it never changes Kafka. Running verify against the same cluster afterwards reports no drift.

Before you start

  • The CLI with a cluster config file, clusters/prod.yaml on this page.
  • A connecting principal that may read what you import: Describe and DescribeConfigs on the topics, Describe on the Cluster resource for ACLs and SCRAM credentials, and DescribeConfigs on the Cluster resource for quotas. A super-user reads everything.
  • To import schemas, a schemaRegistry block in the cluster config. To import Confluent role bindings, an authorization.mds block and rbac in authorization.accessBackends (Manage Confluent RBAC); without both, role bindings are not imported.
  • On Confluent Cloud, add --skip-quotas and --skip-users (Supported platforms).

Steps

1. Import into a directory

monedula-gitops import cluster --cluster-config-file ./clusters/prod.yaml --output-dir ./imported

The import writes one file per resource under <namespace>/<kind directory>/, for example imported/default/topics/payments.orders.yaml. The kind directories are topics, access, quotas, rolebindings, users and schemas. Every manifest refers to the cluster by the metadata.name of the KafkaCluster in your cluster config, and the import prints a summary with the number of resources of each kind and any warnings. It generates:

  • A KafkaTopic per topic, with its partitions and configuration. Kafka and Confluent housekeeping topics, such as __consumer_offsets, _schemas and _confluent*, are skipped.
  • Producer and consumer ACLs that match a topic exactly become that topic’s access block. Prefixed or ambiguous ACLs become rules of a KafkaAccessPolicy, and the summary warns about them.
  • A KafkaQuota per client quota entity.
  • A KafkaUser per SCRAM username, except the connecting principal’s own credential.
  • Schema subjects named <topic>-value and <topic>-key, folded into the topic’s spec.schema, with each schema body written verbatim under schemas/. A subject named <topic>-<recordFullName>, where the record name matches its schema body, becomes the topic’s value schema with subjectStrategy: TopicRecordName. If two record-based subjects map to one topic, the import keeps one and the summary warns about the other.
  • The live role bindings, when imported. A DeveloperWrite on a topic becomes a producer in that topic’s access block, and a consumer folds in only when the principal has exactly one topic DeveloperRead and one group DeveloperRead. Everything else, such as ResourceOwner, cluster-scoped roles, Schema Registry, Connect and ksqlDB scopes, custom roles and prefixed patterns, becomes a KafkaRoleBinding.

If the folded access blocks would not reproduce the live ACLs and role bindings exactly, the import writes every ACL as a KafkaAccessPolicy rule and every role binding as a KafkaRoleBinding instead, and the summary warns.

Without --output-dir, the import writes the manifests to stdout and the summary to stderr, so you can pipe the manifests. It refuses stdout when the cluster has schema subjects to import, because schema bodies must be written as files.

2. Choose what to import

--skip-schemas, --skip-users and --skip-quotas leave out a kind that something else owns, such as schemas that producer pipelines register, or that the cluster cannot serve, such as quotas on Confluent Cloud; --skip-schemas also avoids the Schema Registry connection. --include-internal adds the housekeeping topics. --include-connecting-user adds the connecting principal’s own SCRAM credential, which is skipped by default because managing it in a manifest risks rotating the credential monedula-gitops itself connects with. To import everything except users:

monedula-gitops import cluster --cluster-config-file ./clusters/prod.yaml --output-dir ./imported --skip-users

3. Choose namespaces and overwrite behavior

--namespace-strategy decides the namespace of each imported topic, and so the directory it is written to: single (default) uses --namespace (default default), prefix uses the first segment of the topic name before --prefix-separator (default .), regex uses the one capture group of --namespace-regex, and mapping-file reads a YAML or JSON map from topic to namespace from --namespace-mapping-file. An access policy follows the namespace of the topics its rules name.

When you import again into an existing directory, --overwrite decides what happens to files that already exist: never, the default, skips them, changed rewrites a file only when its generated content differs, and always rewrites every file.

4. Finish the manifests

Review the summary and the generated files before you commit them:

  • Each imported KafkaUser reads its password from a placeholder environment variable named KAFKA_USER_<SANITIZED_USERNAME>_PASSWORD, because Kafka never returns SCRAM passwords; the summary warns that they are not recoverable. Set the variables, or switch the manifests to another password source, before you apply them. A user with credentials under both SCRAM mechanisms is imported with SCRAM-SHA-512, and the summary warns about the SCRAM-SHA-256 one, which verify and apply then leave alone.
  • Subjects named after the record alone, the RecordName strategy, cannot be tied to a topic from cluster state. The summary lists them for manual attribution, and they are never written into a manifest. Other subjects that match no topic appear in a warning.
  • Check the KafkaAccessPolicy rules that the summary warns about. They hold the ACLs that could not be folded into a topic’s access block, such as prefixed patterns.
  • The manifests are written for the CLI: topics read schema bodies with valueFrom.file, and users read passwords with valueFrom.env. The operator resolves neither, so switch schema bodies to inline, configMapKeyRef or secretKeyRef, and passwords to secretKeyRef or generate, before you apply them as custom resources.

Check it worked

The imported files are nested in subdirectories, so verify them with -R:

monedula-gitops verify -f ./imported -R --cluster-config-file ./clusters/prod.yaml   # exits 0: no drift

verify exits 0: the manifests describe the live topics, ACLs, quotas, users, role bindings and schemas, including each schema’s subject, body and compatibility level. Passwords are never part of drift, so the placeholders do not affect the result.