Monedula

Scenario 01.1.4 · acks=1 durability gap

Kafka acks=1 data loss: min.insync.replicas does not help

Producers & EOS · Intermediate · about 3 min · Apache Kafka 4.3

A producer with acks=1 is acked by the leader, which crashes before any follower replicates - the acknowledged write is silently lost.

Run this scenario in the playground

Consumers never saw it. t+8.000s

The consumer has polled repeatedly and read nothing past offset 0 - the acknowledged record is permanently lost from every consumer. acks=all (with min.ISR=2) would have held the ack until every in-sync replica, all three here, had the record.

Open at this step: step 6, Consumers never saw it

What you start with

The topic looks durable - RF=3, min.insync.replicas=2 - but acks=1 never consults min.ISR (that gate only applies to acks=all), so the leader acks alone. Here the leader is killed while the record sits only on it; because the high watermark never advanced, the new leader comes up with nothing to recover and no consumer ever sees the write.

Step by step

  1. Durable-looking topic - RF=3, min.ISR=2 t+0.000s payments has replication.factor=3 and min.insync.replicas=2. The topic config looks safe, but durability ultimately depends on the producer's acks setting. Open at this step: step 1, Durable-looking topic - RF=3, min.ISR=2
  2. acks=1 - leader acks after a local write t+2.500s The producer is acknowledged ('ok') the moment the leader appends the record to its own log (LEO=1). The high watermark is still 0: no follower has replicated the record, and acks=1 never waited for them. Open at this step: step 2, acks=1 - leader acks after a local write
  3. The gap: acked, but uncommitted t+4.900s The producer believes the write succeeded, yet the record lives on exactly one broker. HW is still 0, so the record is not yet below the high watermark and stays invisible to consumers - and one crash from being lost. Open at this step: step 3, The gap: acked, but uncommitted
  4. Leader killed before replication t+5.200s broker-1 (the leader) is gone. The partition is leaderless and electing. The surviving ISR (broker-2, broker-3) never fetched the record - their LEO is 0. Open at this step: step 4, Leader killed before replication
  5. New leader at LEO 0 - record lost t+6.000s A new leader is elected from the surviving ISR at LEO 0. There is nothing to recover: the only copy was on the dead leader. The high watermark is still 0 and never advances. Open at this step: step 5, New leader at LEO 0 - record lost
  6. Consumers never saw it t+8.000s The consumer has polled repeatedly and read nothing past offset 0 - the acknowledged record is permanently lost from every consumer. acks=all (with min.ISR=2) would have held the ack until every in-sync replica, all three here, had the record. Open at this step: step 6, Consumers never saw it

The takeaway

min.insync.replicas only protects acks=all: with acks=1 an acknowledged write can still be lost if the leader dies before any follower fetches it, so durability guarantees require acks=all alongside RF and min.ISR.

Configs in this scenario

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=01.1.4" title="Kafka Simulator: acks=1 durability gap" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/acks1-durability-gap/">Kafka acks=1 data loss: min.insync.replicas does not help</a></p>

Further reading

Apache Kafka documentation

Troubleshooting

More in Producers & EOS

All scenarios