Scenario 01.3.4 · Out-of-order detection and max.in.flight

Kafka max.in.flight.requests.per.connection and ordering

Producers & EOS · Intermediate · about 3 min · Apache Kafka 4.3

With idempotence OFF, max.in.flight>1 lets a retried earlier batch land after a later one (reorder); max.in.flight=1 (or idempotence ON) preserves produce order.

What you start with

max.in.flight.requests.per.connection caps how many un-acked produce requests may be on the connection at once - and with idempotence OFF it is the ordering knob. Half 1 uses max.in.flight=2: a retried earlier record (A) lands AFTER a later one (B), giving log [A, B, A]. Half 2 drops it to 1, blocking later sends until the retry commits, so order is preserved as [C, C, D].

Step by step

1. p1 - idempotence OFF, max.in.flight = 2 t+0.000s One non-idempotent producer on orders/0. max.in.flight.requests.per.connection = 2 lets more than one produce request be in flight at once - the setting that, with idempotence off, lets a retry reorder the log. Open at this step: step 1, p1 - idempotence OFF, max.in.flight = 2
2. A committed at offset 0 t+1.100s A is produced and commits durably at offset 0 (acks=all). Its ack is now on the way back to the producer. Open at this step: step 2, A committed at offset 0
3. A's ack is lost - p1 retries A (deferred) t+3.000s drop_ack loses A's acknowledgement, so p1 must re-send A. Because max.in.flight > 1, the connection still accepts other requests while this retry waits out retry.backoff.ms - leaving a window for a later batch to slip ahead. Open at this step: step 3, A's ack is lost - p1 retries A (deferred)
4. B commits at offset 1 - ahead of the retried A t+3.300s B is produced during the retry window and commits at offset 1, AHEAD of A's still-pending retry. The log is now [A, B] - but A's retry has not landed yet. Open at this step: step 4, B commits at offset 1 - ahead of the retried A
5. A's retry lands at offset 2 - REORDER t+3.600s A's retry finally appends at offset 2. The log is [A, B, A]: A was produced before B, yet its retried copy sits AFTER B. max.in.flight > 1 + idempotence off reordered the log (and, being non-idempotent, also duplicated A). Open at this step: step 5, A's retry lands at offset 2 - REORDER
6. max.in.flight → 1 t+5.000s A config change drops max.in.flight.requests.per.connection to 1 (live in the simulator; a real producer is recreated with it). Now the connection holds a single in-flight request at a time, so the producer blocks later sends until each retry commits. Open at this step: step 6, max.in.flight → 1
7. C committed at offset 3 t+7.000s C is produced and commits at offset 3. Its ack is in flight, just like A's was. Open at this step: step 7, C committed at offset 3
8. C's ack is lost - the retry re-appends at offset 4 t+8.000s drop_ack loses C's ack. With max.in.flight = 1 the producer cannot have a later request outstanding, so nothing can overtake the retry (the simulator re-appends it in the same tick at offset 4; real Kafka still waits retry.backoff.ms). Open at this step: step 8, C's ack is lost - the retry re-appends at offset 4
9. D commits at offset 5 - order preserved t+8.300s D is produced only after C's retry settled, landing at offset 5. The log tail is [C, C, D]: C's retry stayed ahead of D. max.in.flight = 1 preserved produce order - though C is still duplicated (only idempotence removes that). Open at this step: step 9, D commits at offset 5 - order preserved
10. Final log - reorder then order preserved t+9.200s Full log [A, B, A, C, C, D]. Offsets 0-2: max.in.flight = 2 let A's retry land after B (reorder). Offsets 3-5: max.in.flight = 1 kept C's retry ahead of D (order preserved). max.in.flight controls ORDER; idempotence (01.3.1 / 01.3.3) is what removes the duplicates. Open at this step: step 10, Final log - reorder then order preserved

The takeaway

With a non-idempotent producer, max.in.flight>1 can reorder the log when a retry overtakes a later batch; set max.in.flight=1, or enable idempotence, to keep produce order. Note max.in.flight controls order, not de-duplication - duplicates persist until idempotence is on.

Configs in this scenario

• max.in.flight.requests.per.connection How many unacknowledged requests the producer sends on one connection (default 5); above 1 without idempotence a retried batch can land after a later one, and idempotence needs 5 or fewer. How max.in.flight.requests.per.connection works
• acks How many replicas must have a write before the producer counts it as sent: 0 (none), 1 (the leader only) or all (every in-sync replica, the default, and required for idempotence). How acks works
• retry.backoff.ms How long the producer waits before retrying a failed send, so it does not hammer a broker that is still recovering.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=01.3.4" title="Kafka Simulator: Out-of-order detection and max.in.flight" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/max-in-flight-ordering/">Kafka max.in.flight.requests.per.connection and ordering</a></p>

Further reading

Apache Kafka documentation

Troubleshooting

• Kafka messages out of order: causes and fixes

More in Producers & EOS

• Kafka producer acks=0: fire and forget, silent data loss
• Kafka producer acks=1: acknowledged by the leader only
• Kafka acks=0 vs acks=1 vs acks=all on a degraded ISR
• Kafka acks=1 data loss: min.insync.replicas does not help
• Kafka linger.ms vs batch.size: when the producer sends
• Kafka producer partitioners: keyed, round-robin and sticky
• Kafka compression.type: fewer, larger producer batches
• Kafka buffer.memory and max.block.ms: producer backpressure
• Kafka idempotent producer: retrying NOT_ENOUGH_REPLICAS
• Kafka enable.idempotence=false: a retry writes a duplicate
• Kafka idempotent producer dedup vs a plain producer

All scenarios