---
title: "Kafka enable.idempotence: no duplicates on retry"
description: "How the Kafka idempotent producer works: producer ids and sequence numbers drop duplicate retries and keep order. Default true since Kafka 3.0. Simulated live."
url: https://monedula.dev/kafka-simulator/configs/enable-idempotence/
---

# enable.idempotence: the idempotent producer

Producer config · default true (since Kafka 3.0) · Apache Kafka 4.3.

enable.idempotence makes the Kafka producer write each record exactly once per partition even when it retries: the broker recognises a retried batch by its producer id and sequence number and drops the copy. It is on by default since Kafka 3.0, and needs acks=all, retries above 0 and max.in.flight.requests.per.connection of 5 or less.

## At a glance

- Config: `enable.idempotence` (producer)
- Default: true (since Kafka 3.0)
- Valid values: true, false
- Reference: [Apache Kafka documentation](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_enable.idempotence)

## What it does

A producer that gets no answer to a request cannot tell whether the write failed or only the acknowledgement was lost, so it retries. Without idempotence, a retry of a write that did succeed appends a second copy. With idempotence, the producer gets a producer id from the broker and numbers each batch per partition; the broker remembers the last sequence numbers it accepted, acknowledges a duplicate without appending it, and rejects a batch that arrives out of order, which also keeps order with up to 5 requests in flight.

## The trade-off

Idempotence costs almost nothing, which is why it is the default. Its limits are scope: it only covers retries by the same producer instance within one session, not resends by application code or a producer that restarts and replays its input. For those, use transactions with a transactional.id, or make the consumer idempotent. If acks, retries or max.in.flight.requests.per.connection conflict with it and enable.idempotence was not set explicitly, the producer quietly turns it off.

## Advice

- Set enable.idempotence=true explicitly, so that a conflicting setting fails with a ConfigException instead of turning idempotence off silently.
- Keep acks=all and leave retries and delivery.timeout.ms at their defaults.
- For read-process-write pipelines, add transactions and read_committed consumers on top; idempotence alone does not cover them.

## See it in the simulator

- [Kafka idempotent producer dedup vs a plain producer](https://monedula.dev/kafka-simulator/scenarios/idempotence-dedup/) (Idempotent dedup vs duplicate), step 3: Lost ack → p-idem re-sends seq 0 → DEDUPED (no second write). Idempotence: the broker recognises the already-committed sequence and acks without appending. Still one record. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=2)
- [Kafka enable.idempotence=false: a retry writes a duplicate](https://monedula.dev/kafka-simulator/scenarios/the-duplicate-problem/) (The duplicate problem), step 4: Duplicate: O1 now at offsets 0 AND 1. The re-sent copy replicates to the ISR and the high watermark advances to 2. With no sequence number to recognise the re-send, the broker treats it as a brand-new record - the log holds O1 twice. Retries without idempotence cause duplicates. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.2&t=3)
- [Kafka idempotent producer: retrying NOT_ENOUGH_REPLICAS](https://monedula.dev/kafka-simulator/scenarios/idempotence-retry/) (Idempotence retry), step 4: broker-2 back - the in-flight retry commits exactly once. No duplicate: the same record lands at offset 0 once the cluster heals. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.1&t=3)

## Related configs

- [Kafka acks: 0 vs 1 vs all explained](https://monedula.dev/kafka-simulator/configs/acks/)
- [Kafka max.in.flight.requests.per.connection explained](https://monedula.dev/kafka-simulator/configs/max-in-flight-requests-per-connection/)
- [Kafka read_committed vs read_uncommitted (isolation.level)](https://monedula.dev/kafka-simulator/configs/isolation-level/)