---
title: "Kafka consumer group keeps rebalancing: causes and fixes"
description: "Kafka consumer group keeps rebalancing, with CommitFailedException or \"consumer poll timeout has expired\": slow processing, missed heartbeats, restarts."
url: https://monedula.dev/kafka-simulator/troubleshooting/consumer-group-keeps-rebalancing/
---

# The consumer group keeps rebalancing

Kafka symptom.

## Quick answer

A consumer group rebalances whenever a member joins, leaves or is declared dead, so repeated rebalances mean members keep dropping out and coming back. The usual causes are processing that takes longer than max.poll.interval.ms, heartbeats missing for longer than the session timeout, and restarts of members without a static group.instance.id. A member removed for a slow poll loop logs "consumer poll timeout has expired", and its next commitSync() fails with CommitFailedException. Find which one applies before raising any timeout.

## What it means

Each partition in a group is owned by exactly one member at a time, and the group coordinator reassigns partitions when membership changes or when a subscribed topic gains partitions. A member is removed in two ways: it stops heartbeating for longer than the session timeout, or it does not call poll() within max.poll.interval.ms, in which case the client leaves the group on its own and logs "consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages." (the KIP-848 consumer writes "Consumer poll timeout has expired."). On the classic protocol, a commitSync() from a member that was removed fails with CommitFailedException: "Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member." The KIP-848 consumer reports the same situation as "OffsetCommit failed with stale member epoch." or "OffsetCommit failed with unknown member ID." On the classic protocol, still the client default as of Kafka 4.3, every removal and every rejoin is a rebalance, and with the default RangeAssignor every member revokes all of its partitions first, so the whole group stops consuming while it runs. The consumer group protocol from KIP-848 (group.protocol=consumer, generally available since Kafka 4.0) is incremental and moves only the partitions that change owner, but a member that keeps timing out still causes churn.

## Likely causes, most common first

1. **Processing takes longer than max.poll.interval.ms.** If more than max.poll.interval.ms (5 minutes by default) passes between two poll() calls, the consumer leaves the group and its partitions move. Its next poll() rejoins, the same slow batch arrives again, and the cycle repeats. A commitSync() after the eviction fails with CommitFailedException, so that batch is processed a second time. A large max.poll.records (500 by default) combined with slow per-record work is the usual trigger. A static member (group.instance.id set) is the exception: it stops heartbeating but keeps its partitions until the session timeout expires, and only then are they reassigned.
2. **Heartbeats stop for longer than the session timeout.** Long garbage collection pauses, CPU throttling in a container, or network trouble between the client and the group coordinator delay heartbeats. On the classic protocol the limit is session.timeout.ms (45 seconds by default, with heartbeat.interval.ms at 3 seconds). With group.protocol=consumer both are set on the broker, by group.consumer.session.timeout.ms and group.consumer.heartbeat.interval.ms.
3. **Deploys and restarts of dynamic members.** Without group.instance.id, a consumer that shuts down leaves the group, and when it starts again it joins as a new member, so a rolling deploy triggers at least one rebalance per instance (KIP-345). Autoscalers that keep adding and removing consumers have the same effect.
4. **Every rebalance stops the whole group.** With the eager RangeAssignor a rebalance revokes every partition from every member. That does not cause more rebalances on its own, but it turns each one into a full stop of consumption, which is usually what makes the problem visible as lag.

## How to fix it

- Confirm the pattern first. kafka-consumer-groups.sh --bootstrap-server localhost:9092 --describe --group my-group --members lists the current member ids, and --state shows whether the group is Stable. On the client, rebalance-rate-per-hour and last-rebalance-seconds-ago show how often it happens, and time-between-poll-max shows how close the gaps between polls get to max.poll.interval.ms.
- If poll gaps are the cause, lower max.poll.records so a batch fits the interval, make processing faster, or raise max.poll.interval.ms to cover the real worst case. When processing time is unpredictable, the KafkaConsumer Javadoc recommends handing records to another thread and pausing the partitions so poll() keeps being called.
- If heartbeats are the cause, fix the pause itself (GC settings, CPU limits) before touching timeouts, and keep heartbeat.interval.ms at no more than a third of session.timeout.ms.
- For deploys, give every instance a stable, unique group.instance.id. A static member does not send a leave request when it shuts down, and if it returns within the session timeout it gets its old partitions back with no rebalance.
- Move off the eager assignor. CooperativeStickyAssignor is already the second entry in the default partition.assignment.strategy, so one rolling bounce that removes RangeAssignor from the list switches the group to cooperative rebalancing. Alternatively, move the group to group.protocol=consumer, where the broker computes assignments.

## What not to do

- Setting session.timeout.ms or max.poll.interval.ms very high to stop the rebalances. A consumer that has really died then keeps its partitions for that long, and a large max.poll.interval.ms delays every rebalance, because a member only joins one inside poll().

## Configs involved

- [`max.poll.interval.ms`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_max.poll.interval.ms): The longest a group member may go between poll() calls before it counts as failed and the group rebalances its partitions to other members (default 300000, 5 minutes); a static member keeps them until its session times out.
- [`max.poll.records`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_max.poll.records): The most records one poll() returns (default 500); a smaller value shortens the processing between two polls, which keeps a group member inside max.poll.interval.ms.
- [`session.timeout.ms`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_session.timeout.ms): How long the group coordinator waits for a heartbeat from a classic-protocol consumer before it removes the member and rebalances (default 45000, 45 seconds); with group.protocol=consumer the broker group.consumer.session.timeout.ms applies instead.
- [`heartbeat.interval.ms`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_heartbeat.interval.ms): How often a classic-protocol consumer sends a heartbeat to the group coordinator (default 3000, 3 seconds); keep it at no more than a third of session.timeout.ms.
- [`group.instance.id`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_group.instance.id): Makes the consumer a static member: after a restart within the session timeout it gets its partitions back without a rebalance, and only one member with this id can be in the group at a time.
- [`partition.assignment.strategy`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_partition.assignment.strategy): The assignors a classic-protocol consumer supports, in order of preference; the default [RangeAssignor, CooperativeStickyAssignor] assigns by range, and CooperativeStickyAssignor moves only the partitions that change owner instead of revoking every partition on each rebalance.
- [`group.protocol`](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_group.protocol): The rebalance protocol of the consumer: classic (the default as of Kafka 4.3) or consumer, the KIP-848 protocol where the broker computes the assignment and the session timeout and heartbeat interval are broker settings.

## Reproduce it in the simulator

- [Kafka max.poll.interval.ms: evicting a stuck consumer](https://monedula.dev/kafka-simulator/scenarios/max-poll-interval/) (Stuck consumer & max.poll.interval.ms): [open at the moment it happens](https://monedula.dev/kafka-simulator/playground/?scenario=02.3.2&t=3)
- [Kafka consumer crash and session.timeout.ms](https://monedula.dev/kafka-simulator/scenarios/consumer-crash/) (Consumer crash & session timeout): [open at the moment it happens](https://monedula.dev/kafka-simulator/playground/?scenario=02.3.1&t=3)
- [Kafka static membership: group.instance.id avoids rebalances](https://monedula.dev/kafka-simulator/scenarios/static-membership/) (Static membership: avoid rebalance): [open at the moment it happens](https://monedula.dev/kafka-simulator/playground/?scenario=02.3.3&t=5)
- [Kafka eager vs cooperative rebalancing (cooperative-sticky)](https://monedula.dev/kafka-simulator/scenarios/rebalance-protocols/) (Rebalance: eager vs cooperative): [open at the moment it happens](https://monedula.dev/kafka-simulator/playground/?scenario=02.2.4&t=2)

## Sources

- [Apache Kafka 4.3: consumer config max.poll.interval.ms](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_max.poll.interval.ms)
- [Apache Kafka 4.3: consumer config session.timeout.ms](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_session.timeout.ms)
- [Apache Kafka 4.3: consumer config group.instance.id](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_group.instance.id)
- [Apache Kafka 4.3: consumer config partition.assignment.strategy](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_partition.assignment.strategy)
- [Apache Kafka 4.3: consumer rebalance protocol (KIP-848)](https://kafka.apache.org/43/operations/consumer-rebalance-protocol/)
- [Apache Kafka 4.3 Javadoc: KafkaConsumer, detecting consumer failures](https://kafka.apache.org/43/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html)
- [Apache Kafka 4.3: consumer group metrics](https://kafka.apache.org/43/operations/monitoring/#consumer-group-metrics)
- [KIP-345: static membership](https://cwiki.apache.org/confluence/display/KAFKA/KIP-345%3A+Introduce+static+membership+protocol+to+reduce+consumer+rebalances)
- [KIP-429: incremental cooperative rebalancing](https://cwiki.apache.org/confluence/display/KAFKA/KIP-429%3A+Kafka+Consumer+Incremental+Rebalance+Protocol)
- [Apache Kafka 4.3.0 source: AbstractCoordinator.java (consumer poll timeout has expired)](https://github.com/apache/kafka/blob/4.3.0/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractCoordinator.java)
- [Apache Kafka 4.3.0 source: AbstractHeartbeatRequestManager.java (KIP-848 consumer poll timeout)](https://github.com/apache/kafka/blob/4.3.0/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractHeartbeatRequestManager.java)
- [Apache Kafka 4.3.0 source: CommitFailedException.java (default message)](https://github.com/apache/kafka/blob/4.3.0/clients/src/main/java/org/apache/kafka/clients/consumer/CommitFailedException.java)
- [Apache Kafka 4.3.0 source: CommitRequestManager.java (KIP-848 commit failures)](https://github.com/apache/kafka/blob/4.3.0/clients/src/main/java/org/apache/kafka/clients/consumer/internals/CommitRequestManager.java)

## Related

- [Kafka consumer lag keeps growing: causes and fixes](https://monedula.dev/kafka-simulator/troubleshooting/consumer-lag-keeps-growing/)