Monedula

Scenario 11.0.5 · Share group poison pill

Kafka share group poison pill: delivery limit and archiving

Share groups · Advanced · about 3 min · Apache Kafka 4.3

A record the consumer never acks is redelivered until the delivery cap, then archived automatically - while the rest drain (KIP-932).

Run this scenario in the playground

Poison archived at the cap. t+2.300s

The delivery count reached the limit (3), so on the next lock expiry the broker archives offset 0 automatically - the poison pill is skipped, not retried forever.

Open at this step: step 3, Poison archived at the cap

What you start with

A "poison pill" is a record that repeatedly fails processing. In EXPLICIT ackMode the consumer leaves offset 0 un-acked, so its acquisition lock keeps expiring: each expiry redelivers the record and bumps its delivery count. The share-group delivery limit (deliveryLimit=3) caps how many times a record may be handed out - once the count reaches the limit the broker ARCHIVES the record on the next lock expiry instead of redelivering it again, and the remaining records (acked normally) drain past it.

Step by step

  1. Cluster ready · idle t+0.000s One partition, a share group with a single consumer w1 in explicit ack mode, delivery limit 3. Press ▶. Open at this step: step 1, Cluster ready · idle
  2. Poison redelivered t+1.300s Offset 0 is never acked, so its acquisition lock keeps expiring and the record is redelivered - its delivery count climbs each attempt (now 2). Open at this step: step 2, Poison redelivered
  3. Poison archived at the cap t+2.300s The delivery count reached the limit (3), so on the next lock expiry the broker archives offset 0 automatically - the poison pill is skipped, not retried forever. Open at this step: step 3, Poison archived at the cap
  4. Drained around the poison t+3.800s Offsets 1 and 2 were acked and drained; the poison was archived, not blocking the queue. Open at this step: step 4, Drained around the poison

The takeaway

The delivery limit stops a poison pill from blocking the queue forever - once a record has been delivered the maximum number of times the broker archives it automatically on the next lock expiry, protecting throughput with no manual reject required.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=11.0.5" title="Kafka Simulator: Share group poison pill" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/share-group-poison-pill/">Kafka share group poison pill: delivery limit and archiving</a></p>

Further reading

KIP-932 on the Apache Kafka wiki

More in Share groups

All scenarios