Scenario 11.0.7 · Share group: batch_optimized acquire
Kafka share.acquire.mode: batch_optimized vs record_limit
share.acquire.mode=batch_optimized stops a poll at a producer-batch boundary instead of spanning two batches (KIP-932).
Run this scenario in the playgroundCluster ready · idle. t+0.000s
One partition, two share groups on it: 'rl' (record_limit) and 'bo' (batch_optimized), both with max.poll.records=3 and batchSize=2. Press ▶.
Open at this step: step 1, Cluster ready · idleProducer wrote 3 batches of 2. t+0.580s
p1 flushed three fixed-size batches - offsets {0,1} {2,3} {4,5} - all replicated (HW=6) before either group's first poll.
Open at this step: step 2, Producer wrote 3 batches of 2Poll 1 · boundary stop vs span. t+0.660s
Both groups poll with max.poll.records=3. record_limit ('rl') grabbed offsets 0,1,2 - spanning batch {0,1} into {2,3}. batch_optimized ('bo') stopped at the batch-0 boundary and grabbed only 0,1; its 3rd slot went unused.
Open at this step: step 3, Poll 1 · boundary stop vs spanPoll 2 · bo trails by a batch. t+1.260s
'rl' accepted 0,1,2 and acquired 3,4,5 - fully drained next poll. 'bo' accepted 0,1 and acquired the next whole batch 2,3, again stopping at the boundary - still two records behind.
Open at this step: step 4, Poll 2 · bo trails by a batchPoll 3 · rl done, bo finishing. t+1.860s
'rl' has accepted all six. 'bo' accepted 2,3 and acquired its last batch 4,5 - it needed an extra poll because each acquisition stops at a producer-batch boundary.
Open at this step: step 5, Poll 3 · rl done, bo finishingBoth groups drained. t+2.460s
'bo' accepted 4,5 - all six records acked in both groups. batch_optimized never split a producer batch across two polls; the price was one more poll to drain the same backlog.
Open at this step: step 6, Both groups drainedWhat you start with
A share consumer's acquire mode decides how a single poll carves records out of the log. With records written in producer batches of two, two single-member groups poll the same partition with the same max.poll.records=3: record_limit (the default) grabs three contiguous records even across a batch boundary, while batch_optimized stops at the next producer-batch edge so one acquisition never spans two batches.
Step by step
- Cluster ready · idle t+0.000s One partition, two share groups on it: 'rl' (record_limit) and 'bo' (batch_optimized), both with max.poll.records=3 and batchSize=2. Press ▶. Open at this step: step 1, Cluster ready · idle
- Producer wrote 3 batches of 2 t+0.580s p1 flushed three fixed-size batches - offsets {0,1} {2,3} {4,5} - all replicated (HW=6) before either group's first poll. Open at this step: step 2, Producer wrote 3 batches of 2
- Poll 1 · boundary stop vs span t+0.660s Both groups poll with max.poll.records=3. record_limit ('rl') grabbed offsets 0,1,2 - spanning batch {0,1} into {2,3}. batch_optimized ('bo') stopped at the batch-0 boundary and grabbed only 0,1; its 3rd slot went unused. Open at this step: step 3, Poll 1 · boundary stop vs span
- Poll 2 · bo trails by a batch t+1.260s 'rl' accepted 0,1,2 and acquired 3,4,5 - fully drained next poll. 'bo' accepted 0,1 and acquired the next whole batch 2,3, again stopping at the boundary - still two records behind. Open at this step: step 4, Poll 2 · bo trails by a batch
- Poll 3 · rl done, bo finishing t+1.860s 'rl' has accepted all six. 'bo' accepted 2,3 and acquired its last batch 4,5 - it needed an extra poll because each acquisition stops at a producer-batch boundary. Open at this step: step 5, Poll 3 · rl done, bo finishing
- Both groups drained t+2.460s 'bo' accepted 4,5 - all six records acked in both groups. batch_optimized never split a producer batch across two polls; the price was one more poll to drain the same backlog. Open at this step: step 6, Both groups drained
The takeaway
batch_optimized aligns acquisitions to producer-batch boundaries - cheaper for the broker to track, at the cost of sometimes acquiring fewer records per poll than the record limit allows.
Configs in this scenario
-
share.How a share-group member batches the records it acquires in one poll: record_limit grabs up to max.poll.records of the lowest available offsets, while batch_optimized additionally stops at the next producer-batch boundary so a single acquisition never spans two producer batches - fewer, batch-aligned acquisitions at the cost of sometimes-smaller polls.acquire. mode -
max.The most records one poll() returns (default 500); a smaller value shortens the processing between two polls, which keeps a group member inside max.poll.interval.ms.poll. records
Embed this scenario
Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.
<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=11.0.7" title="Kafka Simulator: Share group: batch_optimized acquire" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/share-batch-optimized/">Kafka share.acquire.mode: batch_optimized vs record_limit</a></p>Further reading
KIP-932 on the Apache Kafka wiki
More in Share groups
- Kafka share groups: queue semantics with KIP-932
- More Kafka consumers than partitions with share groups
- Kafka share group acquisition lock timeout and redelivery
- Kafka share consumer acknowledge: accept, release, reject
- Kafka share group poison pill: delivery limit and archiving
- Kafka share.acknowledgement.mode: implicit vs explicit
- Renew a Kafka share group acquisition lock (KIP-1222)
- Kafka share group lag: SPSO, SPEO and the acked prefix