Scenario 01.2.1 · linger.ms and batch.size

Kafka linger.ms vs batch.size: when the producer sends

Producers & EOS · Intermediate · about 3 min · Apache Kafka 4.3

Records are batched, not sent one by one. A batch flushes on linger.ms (time) or batch.size (full): fewer requests vs a little latency.

Run this scenario in the playground

batch.size reached · 3-record batch flushed at once. t+1.750s

O3, O4, O5 arrived in a burst and hit batch.size=3 (three records here; real batch.size is 16384 bytes by default), so the batch flushed immediately - without waiting out linger.ms (LEO 5). Bigger batches → fewer requests → more throughput, at the cost of a little latency.

Open at this step: step 4, batch.size reached · 3-record batch flushed at once

What you start with

The producer accumulates records into batches rather than sending each one immediately. A batch flushes on one of two triggers: time (linger.ms elapses) or size (it reaches batch.size). With linger.ms=600 and batch.size=3 (the simulator counts batch.size in records; real Kafka counts bytes, 16384 by default), watch O1/O2 wait out the linger window while the O3/O4/O5 burst flushes instantly on the size trigger.

Step by step

Idle · linger.ms=600, batch.size=3 t+0.000s The producer batches records before sending. A batch flushes on linger.ms (time) or batch.size (full; the simulator counts it in records, real Kafka in bytes). Open at this step: step 1, Idle · linger.ms=600, batch.size=3
O1, O2 batching · log still empty t+0.700s O1 and O2 arrived within the linger window, so they wait together in a batch. Nothing is appended yet (LEO 0) - the producer is holding them to send as one request. Open at this step: step 2, O1, O2 batching · log still empty
linger.ms elapsed · 2-record batch flushed t+1.000s No more records arrived, so at linger.ms the batch flushed: O1 and O2 append together (LEO 2) as a single request. Time-triggered flush. Open at this step: step 3, linger.ms elapsed · 2-record batch flushed
batch.size reached · 3-record batch flushed at once t+1.750s O3, O4, O5 arrived in a burst and hit batch.size=3 (three records here; real batch.size is 16384 bytes by default), so the batch flushed immediately - without waiting out linger.ms (LEO 5). Bigger batches → fewer requests → more throughput, at the cost of a little latency. Open at this step: step 4, batch.size reached · 3-record batch flushed at once

The takeaway

Bigger batches mean fewer produce requests and higher throughput, paid for with a little added latency - the central producer tuning trade-off you set with linger.ms and batch.size.

Configs in this scenario

linger.ms How long the producer waits for more records before it sends a batch that is not full (default 5 ms since Kafka 4.0, 0 before). How linger.ms works
batch.size The most bytes the producer puts in one batch for a partition (default 16384); a batch goes out when it is full or when linger.ms runs out, whichever comes first. How batch.size works

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=01.2.1" title="Kafka Simulator: linger.ms and batch.size" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/linger-and-batch-size/">Kafka linger.ms vs batch.size: when the producer sends</a></p>

More in Producers & EOS

All scenarios