---
title: "Kafka high watermark pinned by a slow in-sync replica"
description: "A throttled broker stays in the ISR at LEO 0, pinning the Kafka high watermark at 0 while two replicas hold the data. Once it drops out, the HW jumps to 3."
url: https://monedula.dev/kafka-simulator/scenarios/hw-recompute-on-isr-shrink/
---

# Kafka high watermark pinned by a slow in-sync replica

Scenario 03.1.4 · HW recompute on ISR shrink. Replication · Intermediate · about 3 min · Apache Kafka 4.3.

HW = min(LEO) over the ISR. A lagging in-sync replica pins the HW down; removing it from the ISR makes the HW jump up.

Run it: https://monedula.dev/kafka-simulator/playground/?scenario=03.1.4

## What you start with

The high watermark is min(LEO) across the ISR, so a replica that is still in-sync but lagging holds the HW down even when the other replicas already have the records. broker-3 is throttled while R1-R3 pile up on brokers 1 & 2; the HW stays 0 until broker-3 drops out of the ISR.

## Step by step

1. **broker-3 throttled · still in the ISR** (t+2.000s): broker-3's replica fetch is throttled, but it hasn't lagged long enough to be removed yet - the ISR is still {1,2,3}. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=03.1.4&t=0)
2. **Records pile up · HW pinned at 0** (t+6.100s): R1 and R2 are on brokers 1 & 2 (LEO 2), but broker-3 is stuck at LEO 0 - and it's still in the ISR. HW = min(LEO) over the ISR = 0, so NOTHING is committed or readable, even though two replicas already have the data. A slow in-sync replica holds everyone back. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=03.1.4&t=1)
3. **broker-3 leaves the ISR · HW jumps to 3** (t+8.100s): broker-3 finally lagged past the threshold and dropped out of the ISR. The HW is recomputed over the survivors {1,2}: min(3,3) = 3 - it jumps from 0 to 3 and all three records commit at once. Shrinking the ISR can ADVANCE the high watermark. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=03.1.4&t=2)

## The takeaway

Removing a lagging replica from the ISR recomputes the HW over the survivors and makes it jump up - the committed, readable position is gated by the slowest in-sync replica, not the fastest.

## Further reading

- [Apache Kafka documentation](https://kafka.apache.org/43/design/design/#replication)