Scenario 00.2.1 · LEO vs high watermark
Kafka high watermark vs log end offset (LEO)
LEO is each replica’s log end; the high watermark is the committed line = min(LEO) across the ISR.
Run this scenario in the playgroundIdle · LEO = HW = 0. t+0.000s
One partition, three replicas. Every replica's log end offset (LEO) and the high watermark (HW) start at 0.
Open at this step: step 1, Idle · LEO = HW = 0Leader's LEO races ahead · HW still 0. t+0.900s
Three records are appended to the leader, so its LEO is 3. The followers, paced by a slow replica fetch, are still at 0. The high watermark = min(LEO) across the ISR = 0, so nothing is committed yet.
Open at this step: step 2, Leader's LEO races ahead · HW still 0Followers caught up · HW = 3. t+1.300s
The followers fetched the three records, so every replica's LEO is 3. With all ISR members caught up, the high watermark advances to 3 - all three records are now committed.
Open at this step: step 3, Followers caught up · HW = 3What you start with
Each replica has a log-end offset (LEO) - the next position it will write - while the high watermark is the committed line, equal to the smallest LEO across the ISR. With followers fetching on a delay, watch the leader’s LEO run ahead of the high watermark until the replicas catch up.
Step by step
- Idle · LEO = HW = 0 t+0.000s One partition, three replicas. Every replica's log end offset (LEO) and the high watermark (HW) start at 0. Open at this step: step 1, Idle · LEO = HW = 0
- Leader's LEO races ahead · HW still 0 t+0.900s Three records are appended to the leader, so its LEO is 3. The followers, paced by a slow replica fetch, are still at 0. The high watermark = min(LEO) across the ISR = 0, so nothing is committed yet. Open at this step: step 2, Leader's LEO races ahead · HW still 0
- Followers caught up · HW = 3 t+1.300s The followers fetched the three records, so every replica's LEO is 3. With all ISR members caught up, the high watermark advances to 3 - all three records are now committed. Open at this step: step 3, Followers caught up · HW = 3
The takeaway
A record is committed only once every in-sync replica has it: the high watermark = min(LEO) over the ISR is what makes a write durable, not the leader’s LEO alone.
Embed this scenario
Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.
<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=00.2.1" title="Kafka Simulator: LEO vs high watermark" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/leo-vs-hw/">Kafka high watermark vs log end offset (LEO)</a></p>Further reading
More in Anatomy
- Kafka topics, partitions and offsets explained
- Kafka write path: one record from producer to consumer
- Kafka message keys: how a key picks the partition
- Kafka offsets are per partition, not global
- Kafka replication factor, leader and follower replicas
- Why Kafka consumers cannot read past the high watermark
- Kafka log start offset: why the earliest offset is not 0