---
title: "Tune Kafka throughput: linger.ms, batch.size and zstd"
description: "A six-record burst leaves as one zstd request under linger.ms=600, max.poll.records=2 paces the Kafka consumer, and a Confluent MRC observer trails then drains."
url: https://monedula.dev/kafka-simulator/scenarios/tune-high-throughput-low-latency/
---

# Tune Kafka throughput: linger.ms, batch.size and zstd

Scenario 12.0.5 · Capstone: tune a high-throughput, low-latency pipeline. Capstones · Advanced · about 4 min · Apache Kafka 4.3 · Confluent Platform feature.

Every dial on one MRC topic: linger.ms + batch.size + zstd compression coalesce a burst into one request, max.poll.records paces the consumer (visible lag), and the DC-B async observer trails the high-watermark then drains - throughput, latency, back-pressure, and replication lag tuned together.

Run it: https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5

## What you start with

Every tuning dial on one MRC topic at once. On the producer, linger.ms + batch.size + zstd compression coalesce a six-record burst into a single time-triggered request; on the consumer, max.poll.records=2 paces the drain so lag is briefly visible; and a DC-B async observer trails the high-watermark under load. Watch how the producer, consumer, and replication knobs interact.

## Step by step

1. **Tuned pipeline ready** (t+0.000s): One MRC topic `events`: 2 sync replicas in DC-A + an async observer (broker-3) in DC-B. The producer is tuned for throughput (linger.ms=600, batch.size=4, compression=zstd); the consumer is back-pressured (max.poll.records=2). [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5&t=0)
2. **Burst buffered - linger window open** (t+0.800s): Six records (E1..E6) arrive in a burst. They sit in the producer buffer, not yet sent: the linger.ms=600 window is still open, and zstd shrinks each record so the EFFECTIVE batch is large enough that batch.size=4 does not trigger an early flush. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5&t=1)
3. **One compressed batch flushed - HW climbs** (t+1.100s): At the linger deadline the whole burst flushes as a SINGLE compressed request (one batch_flush, not the 4+2 that batch.size=4 alone would produce). The DC-A sync ISR commits all six and the high-watermark jumps to 6 - fewer requests, higher throughput. The DC-B observer is still at offset 0. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5&t=2)
4. **Consumer paced + observer lags** (t+1.500s): Two dials at once. The consumer drains only 2 records per poll (max.poll.records=2) so it trails the producer - committed offset 2, position 4 of 6 (back-pressure). Meanwhile the async DC-B observer still sits at 0 while the HW is 6 (observer lag 6) - it replicates asynchronously and never counts toward acks=all. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5&t=3)
5. **Drained - consumer and observer caught up** (t+2.200s): The burst is over: the consumer has worked through its 2-per-poll backlog to offset 6, and the observer has asynchronously caught up to the high-watermark (lag 0). Throughput, latency, back-pressure, and replication lag have all settled. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=12.0.5&t=4)

## The takeaway

Throughput, latency, consumer back-pressure, and cross-DC replication lag are separate dials - big compressed producer batches do not help if max.poll.records throttles the consumer or the observer cannot keep up, so you tune the whole pipeline, not one knob.

## Configs in this scenario

- [`linger.ms`](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_linger.ms): How long the producer waits for more records before it sends a batch that is not full (default 5 ms since Kafka 4.0, 0 before). [How linger.ms works](https://monedula.dev/kafka-simulator/configs/linger-ms/)
- [`batch.size`](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_batch.size): The most bytes the producer puts in one batch for a partition (default 16384); a batch goes out when it is full or when linger.ms runs out, whichever comes first. [How batch.size works](https://monedula.dev/kafka-simulator/configs/batch-size/)
- [`max.poll.records`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_max.poll.records): The most records one poll() returns (default 500); a smaller value shortens the processing between two polls, which keeps a group member inside max.poll.interval.ms.
- [`acks`](https://kafka.apache.org/43/configuration/producer-configs/#producerconfigs_acks): How many replicas must have a write before the producer counts it as sent: 0 (none), 1 (the leader only) or all (every in-sync replica, the default, and required for idempotence). [How acks works](https://monedula.dev/kafka-simulator/configs/acks/)

## Further reading

- [Apache Kafka documentation](https://kafka.apache.org/43/configuration/producer-configs/)