Your first ACL to RBAC migration

Walk through a complete migration on a local Confluent Platform: extract ACLs, plan role bindings, apply them, and verify effective access.

Run the repository’s live-mds-migration example one command at a time, against a single-node Confluent Platform in Docker with RBAC and the Metadata Service (MDS) turned on. Nothing is mocked: apply and verify talk to a real MDS, and extract and the deletion script to a real broker.

What you need

Docker with the Compose plugin. Check with docker compose version. The first run pulls the cp-server image, about 1 GB, and builds an image with the CLI, so allow a few minutes.
Git and a bash shell. The commands on this page use bash syntax.

You need no Go and no local install of the CLI: the example runs it in a container.

1. Get the example

Clone the repository and change into the example directory:

git clone https://github.com/monedula-dev/monedula-acl-rbac-converter
cd monedula-acl-rbac-converter/examples/live-mds-migration

Run the commands on your machine from this directory. It contains:

examples/live-mds-migration/
  docker-compose.yml   # OpenLDAP, a cp-server broker with RBAC and MDS, and the CLI image
  seed-acls.sh         # creates the legacy ACLs on the broker
  client.properties    # Kafka client settings the CLI connects with
  secrets/             # throwaway local credentials for MDS and LDAP
  pipeline.sh, run.sh  # the scripted version of this tutorial

2. Start Confluent Platform

docker compose up -d openldap kafka

This starts two containers: OpenLDAP, which holds the mds admin user, and a cp-server 8.2.1 broker in KRaft mode that also runs MDS on port 8090. Wait until the broker reports healthy, which means MDS answers requests:

docker compose ps kafka --format '{{.Health}}'

Repeat the command until it prints healthy.

3. Create the legacy ACLs

bash seed-acls.sh

The script runs kafka-acls inside the broker container as the superuser, creates these ACLs and ends by listing them:

Principal	ACLs
`User:svc-orders-consumer`	Read and Describe on topic `orders`; Read and Describe on group `orders-consumers`
`User:svc-payments-producer`	Write and Describe on topic `payments`
`User:svc-analytics`	Read and Describe on every topic with the prefix `events.`
`User:svc-fulfillment`	Read and Describe on topic `shipments`
`User:svc-orders-relay`	Read, Write and Describe on topic `orders`
`User:legacy-importer`	Deny Read on topic `pii-events`

4. Open a shell with the CLI

docker compose run --rm converter

The converter service is an image with monedula-acl-rbac on the PATH, built from the repository the first time you use it. It opens a bash shell in /work, the example directory mounted from your machine, on the same network as the broker. Run steps 5 to 10 in this shell.

Set the addresses and the MDS credentials that the next commands use, and create the run directory:

MDS=http://kafka:8090
BROKER=kafka:9094
MDS_AUTH=(--mds-url "$MDS" --mds-user mds --mds-password-file secrets/mds.pw)
mkdir -p out

out/ is the run directory: every command writes its artifacts there. MDS_AUTH holds the MDS flags; the CLI exchanges the user name and password for an MDS token and keeps the token in memory only.

5. Write scopes.yaml

Every role binding is scoped to the ID that MDS knows the Kafka cluster by. Ask MDS for it and write it to out/scopes.yaml:

CID="$(curl -sf "$MDS/v1/metadata/id" | sed -E 's/.*"id" *: *"([^"]+)".*/\1/')"
printf 'kafka_cluster: %s\n' "$CID" > out/scopes.yaml
cat out/scopes.yaml

The broker starts with a fixed cluster ID, so the file contains kafka_cluster: MkU3OEVBNTcwNTJENDM2Qk, the same value as the example’s committed scopes.yaml. For a real cluster, Find the cluster IDs for scopes.yaml shows where the ID comes from.

6. Extract the ACLs

monedula-acl-rbac extract --from live \
  --bootstrap-server "$BROKER" --command-config client.properties \
  --out out/acls.json

--from live reads the ACLs from the running broker. client.properties holds the Kafka client settings in the same format that kafka-acls.sh --command-config takes:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="kafka" password="kafka-secret";

extract writes out/acls.json, the canonical ACL snapshot, with one row per principal, operation and resource, each with a numeric id that later steps refer to.

7. Plan the role bindings

monedula-acl-rbac plan --acls out/acls.json --scopes out/scopes.yaml \
  --out out/plan.json --allow-rejected

plan applies the built-in mapping rules and writes out/plan.json, its checksum out/plan.json.sha256, and a readable report, out/report.txt. --allow-rejected lets plan exit 0 with the DENY recorded as rejected; without it, plan would exit 3.

Read the report:

cat out/report.txt

The example’s committed copy, expected/report.txt, shows what to look for. Below its first line, which holds the generation time, it reads:

plan: 7 binding(s), 0 unmapped, 1 rejected, 0 warning(s); DENY: 1 safe / 0 would-grant / 0 unknown

Bindings:
  [CREATE] DeveloperRead for User:svc-analytics -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:events. (PREFIXED)
    source_acl_ids: [2 3]
  [CREATE] DeveloperRead for User:svc-fulfillment -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:shipments (LITERAL)
    source_acl_ids: [4 5]
  [CREATE] DeveloperRead for User:svc-orders-consumer -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Group:orders-consumers (LITERAL)
    source_acl_ids: [8 9]
  [CREATE] DeveloperRead for User:svc-orders-consumer -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:orders (LITERAL)
    source_acl_ids: [6 7]
  [CREATE] DeveloperRead for User:svc-orders-relay -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:orders (LITERAL)
    source_acl_ids: [10 11 12]
  [CREATE] DeveloperWrite for User:svc-orders-relay -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:orders (LITERAL)
    source_acl_ids: [10 11 12]
  [CREATE] DeveloperWrite for User:svc-payments-producer -> kafka=MkU3OEVBNTcwNTJENDM2Qk on:
    - Topic:payments (LITERAL)
    source_acl_ids: [13 14]

Rejected:
  - DENY_PERMISSION (source_acl_ids=[1]): DENY Read on Topic:pii-events for User:legacy-importer

DENY analysis:
  - source_acl_id=1 SAFE_TO_REMOVE

Each binding lists the source_acl_ids of the ACLs it replaces, so you can trace it back to acls.json. User:svc-orders-relay gets two bindings, DeveloperRead and DeveloperWrite, the prefixed ACL on events. a prefixed binding, and the DENY none; How ACLs map to RBAC roles explains why. The DENY analysis classifies the DENY SAFE_TO_REMOVE.

8. Apply the bindings

Preview the changes first:

monedula-acl-rbac apply --plan out/plan.json --dry-run "${MDS_AUTH[@]}"

The dry run writes nothing to MDS and records every call it would make, with the request body, in out/would-apply.log. Read that file, then create the bindings:

monedula-acl-rbac apply --plan out/plan.json --confirm "${MDS_AUTH[@]}"

apply writes every MDS call and its result to out/apply.log.

9. Verify effective access

monedula-acl-rbac verify --plan out/plan.json --mode effective "${MDS_AUTH[@]}"

verify asks MDS whether each source ACL’s principal, operation and resource is now allowed, so it also catches a binding that exists but grants nothing (Verify effective access). It writes out/verify.json and exits 0 when every result is EFFECTIVE_OK.

If verify exits 5 right after apply, MDS’s role-binding cache may not have caught up yet: wait a few seconds and run it again.

10. Generate the deletion script

In a real migration, wait at least 24 hours here; see Delete the converted ACLs. In this tutorial, carry on:

monedula-acl-rbac delete-acls --plan out/plan.json --verify out/verify.json \
  --bootstrap-server "$BROKER" --command-config client.properties \
  --principal User:svc-orders-consumer --principal User:svc-payments-producer \
  --principal User:svc-analytics --principal User:svc-fulfillment \
  --principal User:svc-orders-relay \
  --confirm --i-understand-this-is-destructive

delete-acls deletes nothing itself. It writes delete-acls.sh, deleted-acls.json and rollback.sh to out/, covering only source ACLs whose verify result is EFFECTIVE_OK, so never the DENY.

Read the script before you run it:

cat out/delete-acls.sh

Each removal is one kafka-acls --remove --force command that logs OK or FAIL with the ACL’s id, and the script stops at the first failure. Its header records the checksums of plan.json and verify.json.

Leave the converter shell:

exit

11. Run the deletion script

The script calls kafka-acls, which the broker container has. From the example directory on your machine, run it there:

delete_script="$(sed -E \
  -e 's#client\.properties#/tmp/admin.properties#g' \
  -e 's#^LOG=.*#LOG=/tmp/delete.log#' \
  out/delete-acls.sh)"
printf '%s' "$delete_script" | docker compose exec -T -e MONEDULA_SKIP_PLAN_HASH_CHECK=1 kafka bash -s

The two sed expressions point the script at the client settings that seed-acls.sh wrote inside the broker container, and at a log file there. The run directory is not mounted in the broker container, so the script cannot re-hash plan.json; MONEDULA_SKIP_PLAN_HASH_CHECK=1 skips that check.

12. Check the result

List the ACLs that are left on the broker:

docker compose exec -T kafka bash -s <<'LIST'
kafka-acls --bootstrap-server kafka:9094 --command-config /tmp/admin.properties --list 2>/dev/null
LIST

The five converted principals have no ACLs left; their access now comes from the role bindings that verify confirmed. The DENY for User:legacy-importer is still there, because delete-acls never touches DENY ACLs.

13. Clean up

Stop the stack and remove its volumes:

docker compose down -v

The run directory out/ stays on your machine as the migration’s audit trail. ./run.sh --down replays every step of this tutorial, asserts the result and tears the stack down.

Next steps

Plan and review role bindings explains unmapped and rejected ACLs and how to edit a plan.
Remove DENY ACLs handles the DENY you left on the broker.
Extract ACLs from any source covers the other eight places your ACLs can come from.