Scenario 02.3.3 · Static membership: avoid rebalance

Kafka static membership: group.instance.id avoids rebalances

Consumer groups · Intermediate · about 4 min · Apache Kafka 4.3

A dynamic member that stays down past session.timeout.ms is evicted and its partition rebalances onto a survivor; a STATIC member (group.instance.id) that rejoins within session.timeout.ms keeps its partition and resumes in place with no rebalance.

What you start with

A dynamic member that drops out and stays gone past session.timeout.ms is evicted and its partitions rebalance to a survivor. With static membership (group.instance.id), a member that rejoins within session.timeout.ms keeps its partitions and resumes in place. This scenario crashes one member in each group and contrasts the two reactions.

Step by step

1. two groups, two members each t+0.000s Both cg-dyn and cg-stat read the same 2-partition topic with a cooperative-sticky member per partition: member 1 → p0, member 2 → p1. In cg-stat, s2 is STATIC - it carries a group.instance.id (KIP-345); every other member is dynamic. Open at this step: step 1, two groups, two members each
2. warm-up consumed (lag 0) t+0.600s One record per partition has been produced and every member has drained its partition, so both groups sit at lag 0. Nothing distinguishes the static member yet - group.instance.id only matters when a member leaves. Open at this step: step 2, warm-up consumed (lag 0)
3. both p1 owners crash t+1.000s d2 (dynamic) and s2 (static) crash at the same instant. d2 schedules a member_eviction at now + session.timeout.ms (t=3000): if it has not rejoined by then it is removed. s2 is static - the coordinator holds its slot open across the outage, so no rebalance occurs as long as it returns before t=3000. Both keep owning p1, which now stalls. Open at this step: step 3, both p1 owners crash
4. session window: p1 stalled in both t+2.500s Midway through the session window. In BOTH groups the crashed member still owns p1 and its backlog grows (the producer kept writing). p0 keeps draining (d1/s1 are alive). Neither member has been removed yet - the divergence depends on who rejoins in time. Open at this step: step 4, session window: p1 stalled in both
5. static member rejoins in time t+2.900s s2 restarts with the SAME instance.id at t=2800 - within session.timeout.ms of its crash (1800 ms < 2000 ms here; timings are compressed, real brokers require at least 6 s) - so the coordinator’s held slot lets it resume p1 in place with NO rebalance (s1 still owns only p0). d2 is still absent, its eviction one tick away. Open at this step: step 5, static member rejoins in time
6. dynamic member evicted + rebalances t+3.200s d2 never returned, so at t=3000 its session.timeout.ms expires: cg-dyn drops d2 to an evicted ghost and REBALANCES p1 onto the survivor d1. cg-stat already absorbed s2’s restart as a no-op. Same crash, opposite reaction - a static member that rejoined within the timeout kept its partition, while the member that stayed gone past it was reshuffled. Open at this step: step 6, dynamic member evicted + rebalances
7. outcome: rebalance vs retained t+4.500s Final state. cg-dyn: d1 owns [p0,p1] after the rebalance, d2 lingers as an evicted ghost. cg-stat: the assignment never moved - s1 → p0, s2 → p1 - and there is no ghost. Static membership turned a crash-and-restart-within-session.timeout.ms into a no-op for the group. Open at this step: step 7, outcome: rebalance vs retained

The takeaway

Static membership trades fast failure detection for rebalance avoidance: a member can restart or briefly disconnect within session.timeout.ms and resume its exact assignment with no rebalance at all.

Configs in this scenario

• session.timeout.ms How long the group coordinator waits for a heartbeat from a classic-protocol consumer before it removes the member and rebalances (default 45000, 45 seconds); with group.protocol=consumer the broker group.consumer.session.timeout.ms applies instead. How session.timeout.ms works
• group.instance.id Makes the consumer a static member: after a restart within the session timeout it gets its partitions back without a rebalance, and only one member with this id can be in the group at a time.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=02.3.3" title="Kafka Simulator: Static membership: avoid rebalance" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/static-membership/">Kafka static membership: group.instance.id avoids rebalances</a></p>

Further reading

KIP-345 on the Apache Kafka wiki

Troubleshooting

• Kafka consumer group keeps rebalancing: causes and fixes

More in Consumer groups

• Kafka consumer poll loop: subscribe, poll, fetch
• Kafka auto.offset.reset: earliest vs latest vs none
• Kafka committed offsets: enable.auto.commit and restart
• Replay a Kafka topic: consumed records stay in the log
• Kafka consumer group: how partitions split across members
• Kafka consumer group rebalance when a consumer joins
• Kafka partition assignors: range vs round-robin vs sticky
• Kafka eager vs cooperative rebalancing (cooperative-sticky)
• Kafka consumer crash and session.timeout.ms
• Kafka max.poll.interval.ms: evicting a stuck consumer
• Kafka consumer lag: why it grows and how it drains
• Evicted Kafka consumer: how a survivor takes its partition

All scenarios