Scenario 05.3.1 · Tiered storage

Kafka tiered storage: offloading old segments to remote

Storage · Advanced · about 2 min · Apache Kafka 4.3

remote.storage.enable · aged segments leave local disk for the remote tier; localStartOffset rises to mark the boundary while logStartOffset stays put - nothing is deleted, only moved.

Run this scenario in the playground

Boundary moves again - two records local. t+2.000s

A second offload raised localStartOffset to 10: offsets 0-9 are remote, only 10-11 are local. This is the steady state of a tiered topic - the local/remote boundary keeps climbing behind the head of the log while logStartOffset stays at 0. What a consumer sees when it reads below that boundary is 05.3.2; the day total retention finally deletes from the remote tier is 05.3.3.

Open at this step: step 4, Boundary moves again - two records local

What you start with

With remote.storage.enable, a topic’s older log segments are offloaded from local disk to cheaper remote storage while staying fully readable (KIP-405 tiered storage). This scenario is only the offload: twelve records are written, then the old prefix moves to the remote tier in two steps, and you watch localStartOffset climb while logStartOffset never moves.

Step by step

Remote storage on · empty log t+0.000s A delete topic with remote.storage.enable. Two offsets matter on a tiered partition: logStartOffset (the oldest offset still readable anywhere) and localStartOffset (the oldest offset still on the broker's local disk). Both start at 0. Press play to produce twelve records. Open at this step: step 1, Remote storage on · empty log
Twelve records, all local t+0.600s Twelve records produced (HW=12). Nothing has been offloaded yet, so localStartOffset = logStartOffset = 0: the whole log sits on local disk, exactly as it would without tiered storage. Open at this step: step 2, Twelve records, all local
Old prefix offloaded to remote t+1.000s tier_offload moved offsets 0-5 to the remote tier and raised localStartOffset to 6 - only 6-11 remain on local disk. logStartOffset is still 0: every one of the twelve records is still readable, the oldest six just live somewhere cheaper. Nothing was deleted. Open at this step: step 3, Old prefix offloaded to remote
Boundary moves again - two records local t+2.000s A second offload raised localStartOffset to 10: offsets 0-9 are remote, only 10-11 are local. This is the steady state of a tiered topic - the local/remote boundary keeps climbing behind the head of the log while logStartOffset stays at 0. What a consumer sees when it reads below that boundary is 05.3.2; the day total retention finally deletes from the remote tier is 05.3.3. Open at this step: step 4, Boundary moves again - two records local

The takeaway

Tiered storage moves data, it does not delete it: each offload raises localStartOffset (the oldest offset still on local disk) while logStartOffset (the oldest readable offset) stays at 0 - so local disk stops bounding how much history a topic keeps.

Configs in this scenario

remote.storage.enable Turns on tiered storage (KIP-405) for the topic: older log segments are offloaded from local disk to a remote tier once they pass local.retention.ms, and a consumer reading past them hits a remote read. Total data is still bounded by retention.ms - tiering changes WHERE the data lives, not how long it is kept.

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=05.3.1" title="Kafka Simulator: Tiered storage" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/tiered-storage/">Kafka tiered storage: offloading old segments to remote</a></p>

More in Storage

All scenarios