Scenario 03.1.1 · The ISR set

Kafka ISR (in-sync replicas): how replicas leave and rejoin

Replication · Intermediate · about 2 min · Apache Kafka 4.3

The in-sync replica set is every replica keeping up with the leader. Replicas leave it when they die or lag, and rejoin when they catch up.

Run this scenario in the playground

Writes continue on the surviving ISR. t+6.000s

R2 is produced and commits on the two surviving in-sync replicas (HW advances to 2). broker-3, still down, holds only the older R1.

Open at this step: step 3, Writes continue on the surviving ISR

What you start with

The in-sync replica set (ISR) is every replica currently caught up to the leader - the set that acks=all and the high watermark are computed over. Here all three replicas start in-sync, broker-3 is killed so the ISR shrinks to {1,2}, a record is produced while it is gone, then it restarts, back-fills what it missed, and rejoins the ISR.

Step by step

All three replicas in-sync · ISR {1,2,3} t+1.200s R1 is replicated to every broker, so all three replicas are caught up to the leader - the ISR is {1,2,3}. acks=all and the high watermark are computed over this set. Open at this step: step 1, All three replicas in-sync · ISR {1,2,3}
broker-3 dies · ISR shrinks to {1,2} t+3.200s broker-3 was killed, so it can no longer be in-sync - the controller fences it and removes it from the ISR, now {1,2}. That still meets min.insync.replicas=2, so the partition keeps accepting acks=all writes. Open at this step: step 2, broker-3 dies · ISR shrinks to {1,2}
Writes continue on the surviving ISR t+6.000s R2 is produced and commits on the two surviving in-sync replicas (HW advances to 2). broker-3, still down, holds only the older R1. Open at this step: step 3, Writes continue on the surviving ISR
broker-3 restarts and rejoins the ISR t+8.200s broker-3 came back, fetched the record it missed (R2), caught up to the leader, and rejoined the ISR - back to {1,2,3}. ISR membership is dynamic: replicas leave when they die or lag and rejoin when they catch up. Open at this step: step 4, broker-3 restarts and rejoins the ISR

The takeaway

ISR membership is dynamic: a replica drops out when it dies or lags and rejoins once it catches back up - the ISR is always the live picture of which copies are durable.

Configs in this scenario

acks How many replicas must have a write before the producer counts it as sent: 0 (none), 1 (the leader only) or all (every in-sync replica, the default, and required for idempotence). How acks works
min.insync.replicas The fewest in-sync replicas, leader included, that must have a write for an acks=all produce to succeed (default 1); with fewer in the ISR the leader rejects the write with NOT_ENOUGH_REPLICAS. How min.insync.replicas works

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=03.1.1" title="Kafka Simulator: The ISR set" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/the-isr-set/">Kafka ISR (in-sync replicas): how replicas leave and rejoin</a></p>

More in Replication

All scenarios