---
title: "Kafka idempotent producer dedup vs a plain producer"
description: "One lost ack, two Kafka producers: the idempotent retry is deduped, the plain one writes a duplicate, and a dropped batch trips OUT_OF_ORDER_SEQUENCE_NUMBER."
url: https://monedula.dev/kafka-simulator/scenarios/idempotence-dedup/
---

# Kafka idempotent producer dedup vs a plain producer

Scenario 01.3.3 · Idempotent dedup vs duplicate. Producers & EOS · Intermediate · about 3 min · Apache Kafka 4.3.

A lost ack: an idempotent producer is deduped where a plain producer writes a duplicate - and a dropped batch trips OUT_OF_ORDER_SEQUENCE_NUMBER.

Run it: https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3

## What you start with

Two producers face the same lost ack side by side. The idempotent producer stamps each record with a per-partition sequence number, so the broker recognises the retried record and dedups it; the plain producer has no sequence and writes a visible duplicate. A dropped batch then leaves a sequence gap that the broker rejects as OUT_OF_ORDER_SEQUENCE_NUMBER.

## Step by step

1. **Two producers on orders/0 - idempotence ON (p-idem) vs OFF (p-plain)** (t+0.000s) [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=0)
2. **p-idem commits X at offset 0 (producerSeq 0)** (t+1.100s): The broker records last-accepted sequence 0 for p-idem on this partition. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=1)
3. **Lost ack → p-idem re-sends seq 0 → DEDUPED (no second write)** (t+3.100s): Idempotence: the broker recognises the already-committed sequence and acks without appending. Still one record. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=2)
4. **p-plain commits X at offset 1** (t+5.100s) [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=3)
5. **Lost ack → p-plain re-sends → DUPLICATE at offset 2** (t+7.100s): No idempotence: the retry writes a second copy. The same record now sits at offsets 1 and 2. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=4)
6. **Dropped batch left a gap → p-idem hits OUT_OF_ORDER_SEQUENCE_NUMBER (producer stops)** (t+11.100s): A batch the broker never received left a gap after the last accepted sequence, so the broker rejects the next batch with OUT_OF_ORDER_SEQUENCE_NUMBER. The simulator stops the producer here, as a transactional producer would have to; a Kafka 4.x idempotent producer instead bumps its epoch, resets the sequence and retries on its own. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=01.3.3&t=5)

## The takeaway

Idempotence gives exactly-once-per-partition writes: the broker's per-partition sequence numbers dedup retries that a plain producer would duplicate, and detect gaps from a lost batch instead of silently misordering the log.

## Further reading

- [Apache Kafka documentation](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_enable.idempotence)

## Troubleshooting

- [OutOfOrderSequenceException in Kafka: causes and fixes](https://monedula.dev/kafka-simulator/troubleshooting/out-of-order-sequence/)
- [Kafka duplicate messages: causes and fixes](https://monedula.dev/kafka-simulator/troubleshooting/duplicate-messages/)