--- title: "Kafka KRaft quorum: losing a majority of controllers" description: "Lose one of three KRaft voters and Kafka leader elections still work. Lose two and the quorum is gone: the next election freezes until a voter returns." url: https://monedula.dev/kafka-simulator/scenarios/kraft-quorum/ --- # Kafka KRaft quorum: losing a majority of controllers Scenario 06.0.2 · KRaft quorum. KRaft · Intermediate · about 4 min · Apache Kafka 4.3. A KRaft controller is a voter quorum: losing one of three voters keeps a majority (cluster fine), losing two leaves no majority - leader elections freeze - and restoring a single voter brings the majority back and the frozen election completes. Run it: https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2 ## What you start with The KRaft controller is not a single node but a quorum of voters running the __cluster_metadata Raft log, and it can serve only while a strict majority is reachable (2 of 3 here). We kill one voter - majority holds, so a leader election still succeeds - then a second, so the next leader loss leaves its election frozen; then we restore one voter and watch the election complete. ## Step by step 1. **KRaft quorum - 3 voters, c0 active, c1/c2 standby** (t+0.000s): The controller is a quorum, not a single node. Three voters (c0 active, c1/c2 standby) replicate the __cluster_metadata Raft log, and a metadata change commits once a majority of them has it. The quorum can keep serving as long as a strict MAJORITY of voters - 2 of 3 - is reachable. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=0) 2. **Standby c2 killed - 2 of 3 voters, majority holds, c0 active** (t+1.400s): One standby voter, c2, is killed. Two of three voters remain reachable - still a majority - so the quorum holds and c0 keeps serving as active controller. A KRaft cluster tolerates losing a minority of its voters with zero disruption to the control plane. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=1) 3. **Leader broker-1 killed - controller elects broker-2 (quorum intact)** (t+2.400s): Proof the control plane is healthy: kill the partition leader broker-1, and the active controller still runs the election, promoting broker-2 (leaderEpoch 1) and shrinking the ISR to the survivors. With a majority of voters up, leader elections complete normally. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=2) 4. **Fresh write commits - cluster fine with one voter down** (t+3.000s): A fresh acks=all write also commits: the new leader broker-2 plus follower broker-3 satisfy min.ISR=2, so the high watermark advances to 2. Both the data plane and the control plane are fully functional with one of three voters down. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=3) 5. **Second voter c1 killed - only c0 left, no majority, quorum lost** (t+3.400s): Now a SECOND voter, c1, is killed. Only c0 is left alive - 1 of 3 is NOT a majority - so the quorum is lost and there is no active controller (active is undefined). The metadata log can no longer commit changes. Existing leaders keep serving for the moment (broker-2 still leads), but the control plane is frozen. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=4) 6. **Leader broker-2 killed - election frozen, no quorum to elect** (t+4.400s): With no quorum, kill the leader broker-2: the partition goes leaderless (leader=null, electing=true) but the election can never complete - there is no active controller to choose and publish a new leader. The partition stays stuck until a voter is restored (restart_controller) and a majority returns. This is why KRaft uses an odd voter count and needs a surviving majority. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=5) 7. **Voter c1 restored - majority back, frozen election completes** (t+5.000s): c1 is restarted. Two of three voters are reachable again - the majority is back - so c0 is the active controller once more. The frozen election completes immediately: broker-3, the last surviving ISR member (at LEO=HW), is elected leader of orders-0. Nothing was done to the brokers; restoring a single VOTER is what unblocked the data plane. The ISR is now one replica, below min.ISR=2, so acks=all writes wait for a broker to return - but the partition is led and readable again. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=06.0.2&t=6) ## The takeaway Quorum availability, not broker count, is what keeps the control plane alive: with a majority gone there is no active controller and a leaderless partition stays that way - and it is restoring a voter, not a broker, that brings the election back. ## Configs in this scenario - [`acks`](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_acks): How many replicas must have a write before the producer counts it as sent: 0 (none), 1 (the leader only) or all (every in-sync replica, the default, and required for idempotence). [How acks works](https://monedula.dev/kafka-simulator/configs/acks/) - [`min.insync.replicas`](https://kafka.apache.org/43/configuration/topic-configs/#topicconfigs_min.insync.replicas): The fewest in-sync replicas, leader included, that must have a write for an acks=all produce to succeed (default 1); with fewer in the ISR the leader rejects the write with NOT_ENOUGH_REPLICAS. [How min.insync.replicas works](https://monedula.dev/kafka-simulator/configs/min-insync-replicas/) ## Further reading - [Apache Kafka documentation](https://kafka.apache.org/43/operations/kraft/) ## Troubleshooting - [Kafka offline partitions with no leader: causes and fixes](https://monedula.dev/kafka-simulator/troubleshooting/offline-partitions-no-leader/)