Kafka topic config

unclean.leader.election.enable: availability vs data loss

Topic config · default false · Apache Kafka 4.3

unclean.leader.election.enable lets a replica that is not in the ISR become leader when no in-sync replica is left (default false). The partition comes back online, but every record the new leader never received is lost, including records acknowledged to acks=all producers. With it off, the partition stays offline until an in-sync replica returns.

Config: unclean.leader.election.enable (topic)
Default: false
Valid values: true, false
Also set at: Also a broker setting, unclean.leader.election.enable, which is the default for topics that do not set it.
Reference: Apache Kafka documentation for unclean.leader.election.enable

Scenario 03.2.3 · Unclean leader step 6

Unclean election enabled - broker-3 leads, records lost. broker-3 (LEO 2) is promoted unclean. The high-watermark drops from 5 to 2 - 3 committed records are gone. Availability restored at the cost of durability. (The simulator elects at once; in KRaft a dynamically enabled flag waits for the periodic check, 5 min by default, unless you run kafka-leader-election.sh with the unclean option.) Open this step in the full playground

What it does

Only replicas in the ISR, the in-sync replica set, have every committed record, so normally only they can be elected leader. When the last of them fails, the partition has no leader. A replica that dropped out of the ISR earlier may still be alive, but it is missing records. With unclean election enabled, the controller elects it anyway: the high watermark falls back to its log end, the records past that point are gone for good, and the old replicas truncate their logs to match when they return.

The trade-off

false picks durability: no acknowledged record is lost, but a partition can be offline until a broker that was in sync comes back, which may be hours. true picks availability: producers and consumers carry on at once, at the price of silently losing committed data, which also breaks exactly-once guarantees and can leave consumers behind offsets that no longer exist.

Advice

Leave it false, and avoid the situation with replication.factor=3, min.insync.replicas=2 and acks=all, plus racks spread across failure domains.
If a partition is offline and you decide its availability matters more than the lost records, trigger a one-off unclean election for that partition with kafka-leader-election.sh --election-type unclean instead of turning the setting on.
Only consider true for data you can afford to lose, such as metrics.

See it in the simulator

Kafka unclean.leader.election.enable: offline or data loss (Unclean leader) Step 6: Unclean election enabled - broker-3 leads, records lost Open at this step: Kafka unclean.leader.election.enable: offline or data loss
Kafka unclean leader election: permanent loss of acked data (Unclean election: permanent data loss) Step 5: broker-2 killed - stale broker-3 promoted UNCLEAN, committed records lost Open at this step: Kafka unclean leader election: permanent loss of acked data

Related configs

All Kafka configs