Scenario 02.2.4 · Rebalance: eager vs cooperative

Kafka eager vs cooperative rebalancing (cooperative-sticky)

Consumer groups · Advanced · about 4 min · Apache Kafka 4.3

A range group stops the world when a consumer joins; cooperative-sticky moves only what must move.

What you start with

Eager rebalancing (the classic protocol with an eager assignor such as range) revokes every assignment and stops the world when membership changes; cooperative-sticky rebalancing instead revokes only the partitions that must change owners. This scenario runs both on a consumer joining the same group.

Step by step

1. Range group ready t+0.000s Two consumers, range assignor (classic protocol). 4 partitions split into contiguous blocks: c1 owns p0/p1, c2 owns p2/p3. Open at this step: step 1, Range group ready
2. Consuming, lag 0 t+2.500s One record per partition produced and consumed; committed offsets caught up to the high-watermark. Open at this step: step 2, Consuming, lag 0
3. Eager rebalance - stop-the-world t+3.150s c3 joined. Every member revoked EVERY partition (range ⇒ eager). Nobody owns anything, so the in-window writes are not consumed and lag grows. Open at this step: step 3, Eager rebalance - stop-the-world
4. Range reassigned t+3.300s Assign phase: a full range recompute over 3 members - c1 p0/p1, c2 p2, c3 p3. Ownership restored; consumption resumes. Open at this step: step 4, Range reassigned
5. Switched to cooperative-sticky t+5.900s Assignor switched to cooperative-sticky: a real group stays on the classic protocol and now uses cooperative (KIP-429) rebalances, not the KIP-848 consumer protocol (the simulator's event log still tags it "consumer"). Same membership, so the distribution is unchanged - but the next join will be incremental. (Real consumers switch via a rolling restart.) Open at this step: step 5, Switched to cooperative-sticky
6. Cooperative rebalance - only the mover stops t+6.650s c4 joined. Only the single moving partition (p1) is revoked/parked; p0, p2, p3 keep their owners and keep consuming. Open at this step: step 6, Cooperative rebalance - only the mover stops
7. Cooperative assignment settled t+6.900s The parked partition lands on c4. One partition each across the four members; the non-moving partitions never stopped. Open at this step: step 7, Cooperative assignment settled

The takeaway

Cooperative-sticky rebalancing avoids the global stop-the-world pause - only the partitions actually being reassigned are revoked, so the rest of the group keeps consuming throughout the rebalance.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=02.2.4" title="Kafka Simulator: Rebalance: eager vs cooperative" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/rebalance-protocols/">Kafka eager vs cooperative rebalancing (cooperative-sticky)</a></p>

Further reading

KIP-429 on the Apache Kafka wiki

Troubleshooting

• Kafka consumer group keeps rebalancing: causes and fixes

More in Consumer groups

• Kafka consumer poll loop: subscribe, poll, fetch
• Kafka auto.offset.reset: earliest vs latest vs none
• Kafka committed offsets: enable.auto.commit and restart
• Replay a Kafka topic: consumed records stay in the log
• Kafka consumer group: how partitions split across members
• Kafka consumer group rebalance when a consumer joins
• Kafka partition assignors: range vs round-robin vs sticky
• Kafka consumer crash and session.timeout.ms
• Kafka max.poll.interval.ms: evicting a stuck consumer
• Kafka static membership: group.instance.id avoids rebalances
• Kafka consumer lag: why it grows and how it drains
• Evicted Kafka consumer: how a survivor takes its partition

All scenarios