Scenario 02.2.1 · Partitions split across members
Kafka consumer group: how partitions split across members
A consumer group shares a topic by splitting its partitions - each partition is owned by exactly one member.
Run this scenario in the playgroundGroup ready - 2 members, 4 partitions. t+0.000s
The group g1 has two members on the 4-partition topic orders. The range assignor immediately splits the partitions into contiguous blocks: c1 owns p0-p1 and c2 owns p2-p3. Every partition has exactly one owner, so each record is processed by just one member of the group.
Open at this step: step 1, Group ready - 2 members, 4 partitionsEach member reads only its own partitions. t+1.700s
One record was produced to each partition. c1 polls and reads p0 and p1; c2 polls and reads p2 and p3 - in parallel. The group's per-partition positions have all advanced to 1 and lag is 0. Splitting partitions across members is how a consumer group scales read throughput without ever double-processing a record.
Open at this step: step 2, Each member reads only its own partitionsWhat you start with
A consumer group scales reads by dividing a topic's partitions among its members, with the assignor (run by the group's leader consumer in the classic protocol) guaranteeing each partition is owned by exactly one member at a time. Here a two-member group splits a four-partition topic and each member polls only its own partitions.
Step by step
- Group ready - 2 members, 4 partitions t+0.000s The group g1 has two members on the 4-partition topic orders. The range assignor immediately splits the partitions into contiguous blocks: c1 owns p0-p1 and c2 owns p2-p3. Every partition has exactly one owner, so each record is processed by just one member of the group. Open at this step: step 1, Group ready - 2 members, 4 partitions
- Each member reads only its own partitions t+1.700s One record was produced to each partition. c1 polls and reads p0 and p1; c2 polls and reads p2 and p3 - in parallel. The group's per-partition positions have all advanced to 1 and lag is 0. Splitting partitions across members is how a consumer group scales read throughput without ever double-processing a record. Open at this step: step 2, Each member reads only its own partitions
The takeaway
Parallelism within a group is bounded by partition count - each partition has a single owner, so a record is processed once per group and consumers beyond the partition count sit idle.
Embed this scenario
Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.
<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=02.2.1" title="Kafka Simulator: Partitions split across members" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/partitions-split-across-members/">Kafka consumer group: how partitions split across members</a></p>Further reading
More in Consumer groups
- Kafka consumer poll loop: subscribe, poll, fetch
- Kafka auto.offset.reset: earliest vs latest vs none
- Kafka committed offsets: enable.auto.commit and restart
- Replay a Kafka topic: consumed records stay in the log
- Kafka consumer group rebalance when a consumer joins
- Kafka partition assignors: range vs round-robin vs sticky
- Kafka eager vs cooperative rebalancing (cooperative-sticky)
- Kafka consumer crash and session.timeout.ms
- Kafka max.poll.interval.ms: evicting a stuck consumer
- Kafka static membership: group.instance.id avoids rebalances
- Kafka consumer lag: why it grows and how it drains
- Evicted Kafka consumer: how a survivor takes its partition