From Kafka ACLs to Confluent RBAC: A Safe Migration Path with a New Open-Source Tool

When we work with banks and large enterprises, Kafka is rarely a greenfield topic.

More often, Kafka has already been running for years. Teams depend on it, critical data flows through it, and the platform has accumulated a long history of operational decisions, naming conventions, service accounts, topic patterns, and access rules. By the time an organization decides to move to Confluent Platform, Kafka is not just a messaging system anymore. It is part of the production backbone.

That context matters, because migrations in such environments are not only technical projects. They are risk management exercises.

In the first phase of a Confluent migration, the safest approach is usually to change as little as possible. The goal is to move the platform foundation, stabilize it, and only then introduce bigger improvements. Authentication and authorization are part of that story. Existing Kafka installations often rely on Apache Kafka ACLs, because ACLs are the built-in authorization model available in open-source Kafka.

Confluent Platform, however, also provides RBAC: role-based access control. RBAC is usually easier to reason about at organizational scale, fits better with platform governance, and integrates naturally with other Confluent components such as Metadata Service, Control Center, and Confluent for Kubernetes.

So the question becomes: how do you move from an existing ACL-based Kafka estate to Confluent RBAC without turning access control into a high-risk manual migration?

That is exactly why we built monedula-acl-rbac-converter.

Why ACL to RBAC migration is harder than it looks

A small Kafka cluster with a handful of topics and users can be migrated manually. You inspect the ACLs, decide the equivalent RBAC roles, create bindings, verify access, and remove the old ACLs.

But that is not what most mature enterprise environments look like.

In real deployments, you may find hundreds or thousands of ACL rules. Some were created by platform teams. Some by application teams. Some came from automation. Some are old but still used. Some use prefixed topic patterns. Some reference service users that have since been renamed. Some were exported from scripts that nobody has touched in years.

Manual migration in that context has several problems:

it is slow,
it is error-prone,
it is hard to review,
it is hard to repeat,
it is difficult to audit,
and it becomes dangerous when cleanup starts.

Creating new RBAC bindings is only half of the migration. The more delicate part is proving that the new bindings provide the same effective access as the old ACLs, and only then removing the source ACLs.

That is why we did not want a tool that simply “translates a file”. We wanted a migration workflow.

What the converter does

monedula-acl-rbac-converter is a command-line tool that reads existing Kafka ACLs, converts them into a Confluent RBAC role binding plan, and helps operators execute the migration in a controlled, auditable way.

It can read ACLs from multiple real-world sources, including:

a live Kafka cluster,
exported kafka-acls.sh --list output,
JSON, YAML, or CSV dumps,
Strimzi KafkaUser manifests,
Confluent for Kubernetes manifests,
a running Kubernetes cluster,
or a shell script containing kafka-acls --add ... commands.

This last case is particularly useful in brownfield environments. Many teams do not start from a clean export. Instead, they have an old setup script that created ACLs in the first place.

For example, an input script may contain commands like:

kafka-acls.sh \
  --bootstrap-server kafka.example.com:9093 \
  --command-config admin.properties \
  --add \
  --allow-principal User:svc-billing \
  --operation Read \
  --topic billing.events \
  --group billing-consumer

kafka-acls.sh \
  --bootstrap-server kafka.example.com:9093 \
  --command-config admin.properties \
  --add \
  --allow-principal User:svc-billing \
  --operation Describe \
  --topic billing.events

The converter can extract those ACL definitions into a canonical acls.json file. From there, it can produce a reviewed migration plan, emit scripts or manifests, apply bindings directly to Confluent MDS, verify the result, and generate cleanup scripts for old ACLs.

A workflow designed for high-risk changes

Access control migrations need guardrails. A wrong permission can break production workloads. A permission that is too broad can create a security incident. A cleanup step performed too early can cause an outage.

For that reason, the production workflow is explicit and step-based:

1. extract       -> create a canonical ACL snapshot
2. plan          -> convert ACLs into an RBAC migration plan
3. apply dry-run -> preview MDS changes
4. apply         -> create RBAC role bindings
5. verify        -> check effective access
6. wait          -> let users exercise the new path
7. delete-acls   -> generate a deletion script for old ACLs
8. review + run  -> execute the cleanup manually

Each step creates artifacts that can be inspected, reviewed, versioned, or attached to a change request.

The tool intentionally separates planning from applying, applying from verifying, and verifying from deletion. This makes the migration easier to reason about and safer to run in regulated environments.

Example: migrating from an ACL setup script

Let’s assume the starting point is a shell script with kafka-acls.sh --add commands.

1. Extract ACLs

First, extract the ACLs into a canonical JSON representation:

monedula-acl-rbac extract \
  --from script \
  --input setup-acls.sh \
  --vars vars.yaml \
  --out runs/billing-batch-1/acls.json

The optional vars.yaml file is useful when the script contains variables. The tool does not guess unresolved values. If variables are present, they should be provided explicitly or the script should be pre-expanded before migration.

At this stage, the output is still only a snapshot. No external state is changed.

2. Create a migration plan

Next, convert the extracted ACLs into a role binding plan:

monedula-acl-rbac plan \
  --acls runs/billing-batch-1/acls.json \
  --scopes scopes.yaml \
  --rules rules.yaml \
  --principals principals.yaml \
  --out runs/billing-batch-1/plan.json

The scopes.yaml file identifies the Confluent clusters that RBAC bindings should target. Optional rules.yaml and principals.yaml files let operators adapt the conversion to their environment, for example by mapping source principals to the principal format expected by MDS.

The planning step also generates a report. This is one of the most important review points in the workflow. It shows what each source ACL became, and flags anything that could not be mapped or should not be converted automatically.

3. Read the report

Before anything is applied, operators should read the generated report.

This is where migration work becomes visible. Instead of relying on a black-box conversion, the team can review the generated bindings, investigate unmapped ACLs, and decide whether custom mapping rules are needed.

DENY ACLs deserve special attention. Confluent RBAC does not have equivalent deny semantics, so DENY ACLs are not silently converted. They are reported and handled separately.

4. Dry-run the apply step

Before creating role bindings in MDS, run a dry-run:

monedula-acl-rbac apply \
  --plan runs/billing-batch-1/plan.json \
  --mds-url https://mds.example.com \
  --mds-token-file ~/.confluent/token \
  --dry-run

The dry-run previews the MDS calls that would be made. It also performs read checks so existing bindings can be skipped idempotently.

This gives operators another checkpoint before the first real change.

5. Apply RBAC bindings

Once the dry-run output is reviewed, apply the plan:

monedula-acl-rbac apply \
  --plan runs/billing-batch-1/plan.json \
  --mds-url https://mds.example.com \
  --mds-token-file ~/.confluent/token \
  --confirm

The apply step creates RBAC bindings in Confluent MDS. It is designed to be re-runnable: if a binding already exists, it is skipped.

6. Verify effective access

After applying RBAC bindings, verify that they actually grant the access originally provided by the ACLs:

monedula-acl-rbac verify \
  --plan runs/billing-batch-1/plan.json \
  --mds-url https://mds.example.com \
  --mds-token-file ~/.confluent/token

This is stronger than checking whether a binding exists. The goal is to confirm effective access for the original principal, operation, and resource.

That distinction matters in enterprise environments where identity propagation, group membership, LDAP integration, or scope mismatches can cause a binding to exist but not behave as expected.

7. Generate ACL deletion scripts

Only after successful verification, and after an operator-defined cooldown period, should source ACLs be removed.

The tool does not delete ACLs directly. Instead, it generates a script that operators inspect and run themselves:

monedula-acl-rbac delete-acls \
  --plan runs/billing-batch-1/plan.json \
  --verify runs/billing-batch-1/verify.json \
  --bootstrap-server kafka.example.com:9093 \
  --command-config admin.properties \
  --principal User:svc-billing \
  --confirm \
  --i-understand-this-is-destructive

The generated artifacts include:

delete-acls.sh
deleted-acls.json
rollback.sh

This is a deliberate human checkpoint. The operator can open the script, review every kafka-acls --remove command, run it for one principal at a time, and keep the rollback script until the migration is stable.

Different outputs for different operating models

Not every organization wants the same apply model.

Some teams are comfortable applying directly to MDS. Others need every change to go through a change-control process, GitOps flow, or external CD pipeline.

The converter supports those scenarios by emitting different artifact types:

# Shell script with Confluent CLI role-binding commands
monedula-acl-rbac emit \
  --plan runs/billing-batch-1/plan.json \
  --format script \
  --out-dir emit/

# Confluent for Kubernetes manifests
monedula-acl-rbac emit \
  --plan runs/billing-batch-1/plan.json \
  --format cfk \
  --out-dir emit/

# curl commands against MDS REST API
monedula-acl-rbac emit \
  --plan runs/billing-batch-1/plan.json \
  --format mds-curl \
  --out-dir emit/

This makes the tool useful both for hands-on migration work and for environments where infrastructure changes must pass through a separate approval and deployment system.

Built for real migration messiness

We tried to support the kinds of inputs and constraints that show up in actual Kafka estates, not only clean demo cases.

That includes:

live Kafka extraction,
text exports,
structured files,
Kubernetes-native resources,
existing Confluent for Kubernetes manifests,
old setup scripts,
principal remapping,
custom conversion rules,
dry-run execution,
direct MDS apply,
emitted scripts and manifests,
effective-access verification,
deletion script generation,
rollback artifacts,
and status/diff commands for repeatable migration batches.

The result is a tool that can be used both for exploration and for production-grade migrations.

For small experiments, the one-shot convert command can run extraction, planning, and script emission in a single step:

monedula-acl-rbac convert \
  --from yaml \
  --input acls.yaml \
  --scopes scopes.yaml \
  --rules rules.yaml \
  --principals principals.yaml \
  > bindings.sh

For production, the explicit pipeline is the recommended path because it preserves audit artifacts and review points.

Final thoughts

Migrating from Kafka ACLs to Confluent RBAC is not just a syntax conversion problem.

It is a controlled change to the authorization model of a running data platform. In large organizations, especially in banking and regulated industries, that change needs to be reviewable, repeatable, auditable, and reversible where possible.

We built monedula-acl-rbac-converter because we kept seeing the same pattern: Kafka was already there, ACLs were already there, and teams wanted to adopt Confluent RBAC without manually translating hundreds of rules under migration pressure.

The tool is not meant to hide the migration process. It is meant to make it visible.

Extract the current state. Plan the target state. Review the report. Dry-run the changes. Apply deliberately. Verify effective access. Generate cleanup scripts. Review them. Then remove old ACLs only when you are ready.

That is the kind of workflow we want for access-control migrations.

We built it because it is useful for us. We hope it will be useful for others too.

You can find monedula-acl-rbac-converter on GitHub. If you find a bug or have a suggestion for improvement, feel free to open an issue.