Monedula

Scenario 00.2.2 · Reading up to the HW

Why Kafka consumers cannot read past the high watermark

Anatomy · Intermediate · about 2 min · Apache Kafka 4.3

A consumer can only read committed records (offsets below the HW); an uncommitted record on the leader is invisible until the HW advances.

Run this scenario in the playground

Uncommitted on the leader · consumer reads nothing. t+0.900s

The record is on the leader (LEO = 1) but not yet replicated, so the high watermark is still 0. The consumer polls during this window but reads nothing: a consumer may only read up to the HW. An uncommitted record is invisible - it could still be lost if the leader failed before replicating.

Open at this step: step 2, Uncommitted on the leader · consumer reads nothing

What you start with

Consumers may only read committed records - those below the high watermark. A record sitting on the leader but not yet replicated to the full ISR is uncommitted, so watch it stay hidden from the consumer until replication advances the high watermark past it.

Step by step

  1. Idle · consumer at 0 t+0.000s One record will be produced. The consumer polls continuously; watch what it is allowed to read. Open at this step: step 1, Idle · consumer at 0
  2. Uncommitted on the leader · consumer reads nothing t+0.900s The record is on the leader (LEO = 1) but not yet replicated, so the high watermark is still 0. The consumer polls during this window but reads nothing: a consumer may only read up to the HW. An uncommitted record is invisible - it could still be lost if the leader failed before replicating. Open at this step: step 2, Uncommitted on the leader · consumer reads nothing
  3. Replicated · HW advances to 1 t+1.300s The followers fetched the record and the high watermark advanced to 1. The record is now committed and becomes readable. Open at this step: step 3, Replicated · HW advances to 1
  4. Consumer reads the committed record t+1.700s On its next poll the consumer reads up to the high watermark, advancing its position to 1. Open at this step: step 4, Consumer reads the committed record

The takeaway

Consumers never see uncommitted data: a record becomes readable only after replication advances the high watermark, which prevents reading writes that could still be lost.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=00.2.2" title="Kafka Simulator: Reading up to the HW" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/consumer-cannot-read-past-hw/">Why Kafka consumers cannot read past the high watermark</a></p>

Further reading

Apache Kafka documentation

More in Anatomy

All scenarios