Monedula
Kafka Simulator docs menu

Fidelity

What the Kafka Simulator models, what it approximates and what it leaves out, against its modeled baseline of Apache Kafka 4.3.

Concepts 2 of 3

1 min read

Updated 17 October 2026

Markdown

On this page

The Kafka Simulator page shows the first and last list and links to the middle one; all three follow the release you are using.

Modeled baseline: Apache Kafka 4.3. Not for production-as-is. The playground says the same in the ⋯ menu of its top bar: “modeled baseline · 4.3”.

What it models

  • Producer → leader → follower fetch → HW/LEO advance, across multiple topics
  • Producer tuning - acks, idempotence & max.in.flight ordering, compression (zstd/gzip/snappy), buffer.memory backpressure, byte-rate quotas
  • Transactions & EOS - PIDs, sequences, LSO, zombie fencing
  • ISR membership, leader epoch, unclean leader election, throttled reassignment (replication.throttled.rate)
  • Retention - time- & size-based (retention.bytes), compaction & tombstones, tiered storage with local.retention.ms (KIP-405)
  • Consumer groups (eager & cooperative rebalances), static membership (KIP-345), share groups (KIP-932)
  • Control plane - KRaft quorum, controller epoch & failover, plus the pre-4.0 ZooKeeper control plane for comparison (Kafka 4.x has no ZooKeeper mode)
  • Multi-DC DR - stretched 2.5-DC / MRC, observer promote & demote, active-passive failover & failback, active-active; MirrorMaker 2 offset translation & checkpoints; Cluster Linking (offset-preserving mirror, promote, bidirectional switchover)
  • Schema Registry - subjects, versions, compatibility modes
  • Authorization - ACLs & RBAC (principal / resource / operation, role bindings, deny-by-default)
  • Hard kill, graceful shutdown, restart, slow follower, network partition
  • Real Kafka metric names, directionally correct

The observers of the 2.5-DC stretched sandbox come from Confluent’s Multi-Region Clusters, a Confluent Platform feature rather than part of Apache Kafka.

Cluster Linking, one of the two ways to mirror between regions, is a Confluent Platform feature too.

What it approximates

Some things are modeled by their effect, and some behave differently from Kafka 4.3 on purpose, so that a scenario stays short and readable. Most lines say what the simulator does and what Kafka does instead, a few describe an effect the simulator only approximates, and the scenarios point it out where it matters.

Producer

  • batch.size - counted in records here; Kafka counts bytes (default 16384)
  • Keyless sticky partitioning - the simulator moves to the next partition after each batch; Kafka 4.x moves after batch.size bytes, to a random partition weighted toward brokers with shorter queues
  • Client configs fixed at creation - acks, compression.type and max.in.flight.requests.per.connection on a producer, and partition.assignment.strategy on a group, change while running here; in Kafka they are fixed when the client is created, so you recreate the producer or restart the consumers
  • Sends to a leaderless partition - fail at once here (acks=0 drops the record and reports it sent, acks=1/all give up after their retries); a Kafka producer holds the batch until a leader appears, or until delivery.timeout.ms (default 120 s) expires it with a TimeoutException
  • Byte-rate quotas & compression - modeled by their effect on batch sizes and timing, not through real codecs or wall-clock byte rates

Consumer groups

  • Auto-commit timing - offsets commit on every next poll; Kafka commits in poll() once auto.commit.interval.ms (default 5 s) has passed, so a crashed consumer can replay more records
  • Group protocol label - a cooperative-sticky group is shown on the consumer protocol (KIP-848); in Kafka cooperative-sticky is an assignor of the classic protocol (KIP-429), and on the consumer protocol the broker assigns partitions and partition.assignment.strategy is not used

Transactions

  • Transaction markers - commit and abort markers take no offset here and land on every partition in one step; in Kafka each is a control record with an offset of its own, written partition by partition, so a read_committed consumer can see a commit on one partition before another
  • Transaction timeout - starts at begin_transaction here, fires exactly at the deadline and shows a transaction_timed_out error; Kafka counts from the first partition added, aborts on a periodic coordinator check (every 10 s by default), and the next send or commit from the producer fails with an epoch error
  • Zombie fencing - a fenced producer fails at begin_transaction here, and its epoch stays put across commits; in Kafka beginTransaction() is local, so the fence lands on the next send or commit, and transaction.version 2 bumps the epoch at every commit. Consumer-group fencing of transactional offset commits (KIP-447) is not modeled

Storage

  • Log segments - retention, compaction and tiering act on single records or batches and can reach the newest data at once; Kafka acts on whole segments, never compacts the active segment, and rolls it before deleting or tiering it
  • Terminal disk view - a teaching scale: a record counts as 1 KiB and a segment rolls every 4 records; Kafka sizes records in bytes and rolls a segment at segment.bytes (default 1 GiB) or segment.ms (default 7 days)

Replication & operations

  • Clean-election leader choice - the simulator elects the in-sync replica with the highest log end offset; Kafka elects the first live in-sync replica in assignment order, so the two can choose different leaders
  • Network partitions - modeled at the offset level (on heal a diverged replica truncates back to the high watermark; Kafka truncates where its leader epoch diverges, which can sit above it), not packet loss or latency jitter
  • Leadership in a network split - where no controller quorum is modeled, or each region has its own, leadership moves to the side holding most of the replicas; the Kafka controller fences the brokers it cannot reach and elects a leader from the ISR it can reach
  • Reassignment hand-over - removed replicas, the leader included, leave as soon as a reassignment runs; Kafka keeps them in the replica set and ISR until the added replicas catch up
  • Replication throttles - replication.throttled.rate acts on catch-up speed in simulated time, not on real byte rates
  • Turning on unclean.leader.election.enable - elects an out-of-sync leader at once here; the KRaft controller acts on its next periodic check, or when you run kafka-leader-election --election-type unclean
  • auto.leader.rebalance.enable - off in scripted runs, and in free play it returns leadership as soon as the preferred replica rejoins the ISR; Kafka defaults it on and checks every 300 s (leader.imbalance.check.interval.seconds)

Control plane

  • KRaft controller catch-up - a newly elected controller can be behind and visibly replays the metadata log before it serves; Raft elects only a voter holding every committed record, which serves once it commits a record in its new epoch
  • ZooKeeper controller hand-over - when the controller broker dies, the simulator gives the role to the lowest-id live broker; in Kafka the first broker to recreate the ephemeral /controller znode wins
  • KRaft leadership after a heal - the lowest-id voter takes the active controller role back; Raft has no preferred leader, so the sitting controller keeps it and the returning voter rejoins as a follower

Multi-DC

  • MRC observer demotion - in scripted runs a promoted observer stays a synchronous replica when the sync replicas return (free-play MRC demotes it on recovery by default); Confluent demotes it automatically
  • MirrorMaker 2 failback - modeled as a reverse re-sync, and producers can write to the mirror topic after a failover; MirrorMaker 2 has no failback action, and producers write to local topics
  • MirrorMaker 2 offset translation - exact, record by record, and a group committed past the mirror resumes at the DR high watermark; MirrorMaker 2 keeps sparse offset syncs, so a translated offset can land a little early, and a group resumes from its last translated checkpoint
  • Cluster Linking details - mirror topics carry a source prefix (primary.orders), and promote removes the whole link even with the source unreachable or lagging; Confluent keeps the source topic name by default, and promote stops one mirror topic and needs the source reachable with no lag (failover otherwise)

Timing

  • Event timing - events are discrete and seeded: their relative order is modeled, not exact latency or throughput
  • Compressed timeouts - timings are compressed so a scenario plays in seconds, and some values, such as consumer session timeouts, sit below limits a broker enforces (group.min.session.timeout.ms)

What it does not model

  • GC pauses, OS page cache, network buffers, byte-exact sizes
  • SASL, mTLS, encrypted transport - the auth handshake & wire crypto (ACL/RBAC authorization itself is modeled)
  • Kafka Streams & Connect - core broker/client protocol only
  • Magnitudes of metric movement - calibrated direction, illustrative scale

The Help button in the playground’s top bar shows the same simplifications, with more detail on what is modeled and not modeled, next to the keyboard shortcuts.