Scenario 04.2.5 · read_committed during a leader election

Kafka read_committed during a leader election

Transactions · Advanced · about 3 min · Apache Kafka 4.3

An open transaction holds the LSO below the HW; killing the partition leader elects a new one, but the LSO survives the election - read_committed stays pinned until commit.

What you start with

A transactional producer holds an open transaction on a single partition - pinning the LSO below the high-watermark - when the partition leader is hard-killed and a new leader is elected from the surviving ISR. The open-transaction state is partition state, so it travels with the leadership handoff.

Step by step

1. Cluster ready · two readers t+0.000s orders has 1 partition, RF=3, min.ISR=2, led by broker-1. Group rc is read_committed, ru is read_uncommitted. Nothing written yet. Open at this step: step 1, Cluster ready · two readers
2. begin transaction t+0.100s tx1 opens a transaction (transactional.id tx-app-1). No records yet, so the LSO equals the HW. Open at this step: step 2, begin transaction
3. Produced · uncommitted t+1.200s tx1 wrote two records; both replicate and the HW advances, but the open txn pins the LSO at the txn's first offset. read_uncommitted (ru) reads both; read_committed (rc) is held at the LSO and reads nothing. Open at this step: step 3, Produced · uncommitted
4. Leader killed · electing t+2.000s broker-1 (the partition leader) is hard-killed. The partition is leaderless and electing; the election is still pending. The open txn still holds the LSO down - rc cannot advance. Open at this step: step 4, Leader killed · electing
5. New leader · LSO still held t+2.600s broker-2 is elected from the surviving ISR (leaderEpoch bumped). The open-txn state is partition state, so it survived the election: the LSO is unchanged and rc is still pinned. The election did not stabilize anything. Open at this step: step 5, New leader · LSO still held
6. commit · LSO releases t+3.600s commit_transaction on the new leader removes the open txn and advances the LSO to the high watermark. read_committed finally drains both records - the election never moved the LSO; only the commit did. Open at this step: step 6, commit · LSO releases

The takeaway

A leader election does not advance the LSO: read_committed stays pinned across the entire election window and drains the records only once the transaction commits on the new, stable leader.

Configs in this scenario

• min.insync.replicas The fewest in-sync replicas, leader included, that must have a write for an acks=all produce to succeed (default 1); with fewer in the ISR the leader rejects the write with NOT_ENOUGH_REPLICAS. How min.insync.replicas works
• transactional.id A stable name that turns on transactions for a producer (and implies idempotence); a new producer that calls initTransactions() with the same id fences the older one.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=04.2.5" title="Kafka Simulator: read_committed during a leader election" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/read-committed-during-election/">Kafka read_committed during a leader election</a></p>

Further reading

Apache Kafka documentation

More in Transactions

• Kafka at-most-once vs at-least-once vs exactly-once
• At-least-once delivery in Kafka: where duplicates come from
• Kafka transaction commit vs abort with read_committed
• Kafka isolation.level: read_committed vs read_uncommitted
• Kafka last stable offset (LSO) with concurrent transactions
• Kafka transaction.timeout.ms: the coordinator aborts
• Kafka consume-process-produce loop with exactly-once
• Kafka zombie producer fencing with transactional.id
• Kafka processing.guarantee=exactly_once_v2 end to end

All scenarios