Monedula
monedula-sim-link docs menu

Handle large clusters

When your cluster is bigger than the Kafka Simulator renders: which topics and consumer groups are kept, and how --clamp trims brokers and partitions.

How-to guides 3 of 6

4 min read

Updated 16 October 2026

Markdown

On this page

Most clusters have more topics and consumer groups than the simulator draws. connect keeps the most eventful ones and says which it kept and why. Brokers and partitions over the limits fail by default, and --clamp trims them, the same way on every run.

The limits

A single-datacenter free-play session renders at most 10 brokers, 6 topics besides orders, 24 partitions per topic, 4 consumer groups and, with --producers, 4 producers. The mapping reference has the full rules.

Before you start

Steps

1. Let connect pick the topics

Run connect on the whole cluster. With more than 6 topics besides orders, it keeps the 6 most eventful, prints the link, and warns with what it kept, why, and what it dropped:

warning: selection has 10 topics besides "orders"; free play renders at most 6 extra, so it keeps the most eventful: payments (2 partitions under-replicated or offline), audit-log (lag 1840), inventory (read by 2 groups), events (5120 records), billing (412 records), clicks (idle); dropping notifications, search-index, sessions, user-profiles (name the topics you want with --topics)

Topics are ranked by, in order:

  1. partitions under-replicated or offline: no live leader, or fewer in-sync replicas than replicas;
  2. consumer lag, summed over every group that has committed offsets on the topic;
  3. how many consumer groups read the topic;
  4. retained records, the latest offset minus the earliest, summed over the partitions;
  5. the name, alphabetically.

The same cluster state keeps the same topics on every run. The warning lists at most 10 dropped names, then how many more.

2. Choose some topics yourself

List the topics you want with --topics (Choose which topics to show). Listed topics are always kept. Add --topics-regex to let the tool fill the free slots with the most eventful of the topics it matches:

monedula-sim-link connect --brokers localhost:9092 \
  --topics billing,refunds --topics-regex '.*'

Listing more than 6 topics besides orders in --topics fails, because the tool will not drop a topic you named without asking:

mapping: selection exceeds what the simulator can render (pass --clamp to trim deterministically instead):
  - --topics lists 7 topics besides "orders"; free play renders at most 6 extra - keeping the most eventful: payments (2 partitions under-replicated or offline), audit-log (lag 1840), inventory (read by 2 groups), events (5120 records), billing (412 records), clicks (idle); dropping sessions

With --clamp, it keeps the 6 most eventful of the listed topics instead.

3. Know which consumer groups are kept

Only a group with a committed offset on a shown topic counts toward the 4. With more, connect keeps the groups with the most lag across the shown partitions, then the ones with the most members, then by name:

warning: 6 consumer groups have committed offsets on the rendered topics; free play renders at most 4, so it keeps the ones with the most lag: audit-sink (lag 1840), payments-ledger (lag 37), inventory-api (3 members), inventory-batch (1 member); dropping reports, search

A group that reads none of the shown topics is left out, and one warning lists every such group.

4. Trim brokers and partitions with --clamp

Over 10 brokers, or over 24 partitions in a topic, connect exits 1 and prints no link. The message lists every broker, partition and --topics violation at once:

mapping: selection exceeds what the simulator can render (pass --clamp to trim deterministically instead):
  - topic big has 30 partitions; free play renders at most 24 per topic - keeping partitions 0..23

Add --clamp to print the link, with a warning for each trim:

monedula-sim-link connect --brokers localhost:9092 --clamp
warning: topic big has 30 partitions; free play renders at most 24 per topic - keeping partitions 0..23
https://monedula.dev/kafka-simulator/playground?scenario=free&actions=...&size=...

The tool keeps the 10 lowest broker ids and partitions 0 to 23 of each topic. A replica on a broker that was trimmed away is dropped from its reassignment, with one warning per partition.

5. Know the other limits

--clamp changes none of these.

At most 4 members of each consumer group are shown, the first 4 the cluster reports. This limit never fails: the tool keeps the first 4 and warns, with or without --clamp.

With --producers, at most 4 producers are shown. When more than 4 wrote to the shown topics in the window, the tool keeps the 4 that wrote to the most partitions, then the most recent, and warns which it kept and which it dropped. This limit never fails either.

A link carries at most 2000 actions, the playground’s cap, and at most 2048 characters (Self-validation before printing explains why). Over the action cap connect fails. Over the length it refuses the link, unless you pass --force to print it anyway with a warning. For either, lower --max-records-per-partition (default 8) or narrow the topics.

Check it worked

connect exits 0 and prints one warning for every limit it applied. Add --dry-run to see the result: the brokers:, topics: and groups: tables list only what was kept, and the size: line carries the kept counts.