Scenario 07.1.1 · Scale out + rebalance

Add a Kafka broker and reassign partitions onto it

Operations · Intermediate · about 3 min · Apache Kafka 4.3

Add a broker, then reassign partitions onto it (kafka-reassign-partitions).

What you start with

Adding a broker gives the cluster more capacity, but Kafka never moves existing partitions onto it on its own - you have to rebalance explicitly with kafka-reassign-partitions. We produce a few records, add broker-4, then reassign two partitions onto it; each new replica starts empty, fetches from the leader, and joins the ISR once it has caught up.

Step by step

1. Cluster ready t+0.000s Three brokers, RF 3, min.ISR 2. Leaders spread across broker-1/2/3. Open at this step: step 1, Cluster ready
2. Records produced t+6.000s O1 committed to partition 0, O4 to partition 2 (acks=all). High-watermark 1 on both. Open at this step: step 2, Records produced
3. broker-4 added t+9.000s A fourth broker joins with no partitions - idle until replicas are placed on it. Open at this step: step 3, broker-4 added
4. Partition 0 reassigning t+12.050s kafka-reassign-partitions: broker-4 joins at offset 0 and starts catching up, not yet in ISR. The simulator drops broker-3 now; real Kafka keeps it in the replica set and ISR until broker-4 has caught up, so redundancy never dips. Open at this step: step 4, Partition 0 reassigning
5. Partition 0 rebalanced t+12.500s broker-4 caught up to the leader and joined ISR. Leader (broker-1) unchanged. Open at this step: step 5, Partition 0 rebalanced
6. Partition 2 reassigning t+16.050s broker-4 joins and catches up, and leader broker-3 is dropped so leadership moves to broker-1. Real Kafka keeps broker-3 leading until broker-4 joins the ISR, then elects the first live in-sync replica of the new assignment. Open at this step: step 6, Partition 2 reassigning
7. Partition 2 rebalanced t+16.500s broker-4 is in ISR on both moved partitions; load shifted off broker-3. Open at this step: step 7, Partition 2 rebalanced

The takeaway

Scaling out is two steps: add the broker, then reassign partitions onto it - an empty broker carries no load until you deliberately rebalance data onto it.

Configs in this scenario

• min.insync.replicas The fewest in-sync replicas, leader included, that must have a write for an acks=all produce to succeed (default 1); with fewer in the ISR the leader rejects the write with NOT_ENOUGH_REPLICAS. How min.insync.replicas works
• acks How many replicas must have a write before the producer counts it as sent: 0 (none), 1 (the leader only) or all (every in-sync replica, the default, and required for idempotence). How acks works

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=07.1.1" title="Kafka Simulator: Scale out + rebalance" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/reassign-scale-out/">Add a Kafka broker and reassign partitions onto it</a></p>

Further reading

Apache Kafka documentation

More in Operations

• Remove a Kafka broker safely: drain it first
• kafka-reassign-partitions: change a replica set by hand
• Kafka replication throttle during partition reassignment
• Change Kafka min.insync.replicas at runtime, no restart
• producer_byte_rate: how a Kafka client quota throttles

All scenarios