---
title: "Run a stretched cluster across data centers"
description: "Start a stretched free-play cluster, one Kafka cluster across three data centers, then lose a data center and see whether writes and elections survive."
url: https://monedula.dev/flock/docs/kafka-simulator/how-to/run-a-stretched-cluster/
---
## Steps

### 1. Start the sandbox

In the left rail of the [playground](/kafka-simulator/playground/), set **cluster type** to **3-DC stretched**. On a narrow screen, pick it from the **scenario** menu in the top bar, under **✦ Free play**. The sandbox starts with:

- three data centers (DCs), `dc-a`, `dc-b` and `dc-c`, each holding one broker and one KRaft controller: `broker-1` and `c-a` in `dc-a`, and so on;
- the topic `orders`, with three partitions, replication factor 3 and `min.insync.replicas` 2, each partition led from a different DC;
- the producer `p1`, with `acks=all`, and the consumer group `cg1`.

The **Why** tab ends with a **DR posture** card that reads "Stretched multi-DC · synchronous": RPO "≈ 0", RTO "seconds".

### 2. Look at a data center

Select the `dc-a` band on the canvas. The Inspector opens on **Data center · dc-a**: whether it is **reachable** from the active controller, its brokers and its controllers, with the active one marked.

### 3. Lose one data center

Select **kill · hard dc-a**. `broker-1` and `c-a` go down. The partition `broker-1` led elects a new leader in another DC, and the two surviving controllers keep the quorum. The producer keeps writing. Select a partition of `orders`: its in-sync replicas list two brokers and its leader is outside `dc-a`.

### 4. Lose a second data center

Select the `dc-b` band and **kill · hard dc-b**. With one controller of three left, the quorum is lost and leader elections freeze. Partitions whose leader was in `dc-a` or `dc-b` stay without a leader, and the producer gets `LeaderNotAvailable` errors. The **DR posture** card adds "Quorum is lost right now - leadership is frozen until it returns."

### 5. Recover

Select the `dc-b` band and **restart dc-b**. With two controllers of three back, the quorum returns, the leaderless partitions elect leaders again and the quorum note leaves the **DR posture** card. Restart `dc-a` the same way.


**The 2.5-DC layout.** Set **cluster type** to **2.5-DC stretched** for the layout with a witness. It has two data DCs, `dc-a` and `dc-b`, each with three brokers: two synchronous replicas and one observer, an asynchronous replica that does not count toward the in-sync replicas. A third site, `dc-c`, labelled **dc-c · witness** on the canvas, holds only a controller, so the quorum keeps a majority when either data DC fails. The topic `orders` has four synchronous replicas and `min.insync.replicas` 3.

Kill the witness first: select the **dc-c · witness** band and **kill · hard dc-c**. Two controllers of three remain and writes go on. Restart it, then kill `dc-a`, which holds the partition's leader and the active controller. Leadership moves to `dc-b`, the observer there is promoted into the in-sync replicas, and after a few retries the producer writes again.



The **Failure** tab has DC-level controls too: **partition dc-a** fences a DC off from the others until you select **heal**, **kill dc-a** destroys it, and **kill KRaft · dc-a** stops only its controller while its brokers stay up.


## Related

- [Run an active/passive pair and fail over](/flock/docs/kafka-simulator/how-to/run-active-passive/) is the alternative design: two separate clusters with asynchronous mirroring.
- [Features by release](/flock/docs/kafka-simulator/reference/features/) lists the free-play topologies this release offers.


- The [Stretched 3-DC](/kafka-simulator/scenarios/stretched-3-dc/) and [Losing a whole DC](/kafka-simulator/scenarios/losing-a-whole-dc/) scenario pages script the same kind of data-center loss.