Scenario 13.2.2 · Change a topic config live
Set retention.ms on a live topic with kafka-configs
One key on one topic, from the shell: set retention.ms, watch a retention run delete whole segment files, put the default back, and meet the per-key validation.
Run this scenario in the playgroundA topic with no retention of its own. t+0.000s
Open the terminal with Ctrl and the backtick key (on a phone, the "..." menu). `kafka-configs --describe --entity-type topics --entity-name orders` lists what this topic sets for itself, and retention.ms is not among them: this simulator runs no time-based sweep for a topic that sets neither retention window, so nothing here ages out. `kafka-topics --describe --topic orders` shows the same absence in its Configs column, from a different tool.
Open at this step: step 1, A topic with no retention of its ownNine records, three segment files. t+1.900s
The producer is done and the log is already more than one file. `ls -lh orders-0` shows three segments, each named for the offset it starts at, the last of them still being written and smaller than the two before it. `du -sh orders-0` prices the whole partition directory - worth remembering, because a config is about to change that number.
Open at this step: step 2, Nine records, three segment filesOne key, set from the shell. t+2.600s
One key on one topic, and no restart anywhere. `kafka-configs --entity-type topics --entity-name orders --alter --add-config retention.ms=60000` is the command that did it, and pressing it forks the sandbox onto exactly the same change. `kafka-configs --describe --entity-type topics --entity-name orders` now carries a third key, and `kafka-topics --describe --topic orders` shows it in the Configs column as well: the value lives on the TOPIC, which is why the inspector and a shared link both pick it up.
Open at this step: step 3, One key, set from the shellRetention took a file, not rows. t+5.600s
A retention run has been through the partition, scripted at an offset here rather than waiting out the 60 s window. `ls -lh orders-0` is one segment shorter than it was: the file that started at offset zero is gone whole, index pair and all, rather than trimmed. `kafka-dump-log --files orders-0/00000000000000000000.log` has nothing left to open. `kafka-get-offsets --topic orders --time earliest` says where the log begins now, and `du -sh orders-0` says what the deletion gave back.
Open at this step: step 4, Retention took a file, not rowsThe default is back, and a value that is refused. t+7.600s
The key was DELETED, not zeroed. `kafka-configs --entity-type topics --entity-name orders --alter --delete-config retention.ms` is the command that removed it, and `kafka-configs --describe --entity-type topics --entity-name orders` is back to the two keys it opened with, with `kafka-topics --describe --topic orders` agreeing - which in this simulator means no time-based deletion at all, and on a real broker would mean falling back to log.retention.ms, seven days by default. Deleting the override is not the same as setting the value to zero. One last thing worth pressing: `kafka-configs --entity-type topics --entity-name orders --alter --add-config retention.ms=abc` never reaches the cluster here, because this terminal validates the value per key before dispatching and names the type Kafka declares for it; the real kafka-configs sends it and the broker rejects it with an InvalidConfigurationException.
Open at this step: step 5, The default is back, and a value that is refusedWhat you start with
This scenario is played from the terminal (Ctrl and the backtick key). A topic config is not a restart: kafka-configs writes one key on one topic and the cluster acts on it immediately. Here retention.ms is set on a topic that had none, a retention run then removes an entire segment file from disk, and the key is deleted again - which is not the same as setting it to zero.
Step by step
- A topic with no retention of its own t+0.000s Open the terminal with Ctrl and the backtick key (on a phone, the "..." menu). `kafka-configs --describe --entity-type topics --entity-name orders` lists what this topic sets for itself, and retention.ms is not among them: this simulator runs no time-based sweep for a topic that sets neither retention window, so nothing here ages out. `kafka-topics --describe --topic orders` shows the same absence in its Configs column, from a different tool. Open at this step: step 1, A topic with no retention of its own
- Nine records, three segment files t+1.900s The producer is done and the log is already more than one file. `ls -lh orders-0` shows three segments, each named for the offset it starts at, the last of them still being written and smaller than the two before it. `du -sh orders-0` prices the whole partition directory - worth remembering, because a config is about to change that number. Open at this step: step 2, Nine records, three segment files
- One key, set from the shell t+2.600s One key on one topic, and no restart anywhere. `kafka-configs --entity-type topics --entity-name orders --alter --add-config retention.ms=60000` is the command that did it, and pressing it forks the sandbox onto exactly the same change. `kafka-configs --describe --entity-type topics --entity-name orders` now carries a third key, and `kafka-topics --describe --topic orders` shows it in the Configs column as well: the value lives on the TOPIC, which is why the inspector and a shared link both pick it up. Open at this step: step 3, One key, set from the shell
- Retention took a file, not rows t+5.600s A retention run has been through the partition, scripted at an offset here rather than waiting out the 60 s window. `ls -lh orders-0` is one segment shorter than it was: the file that started at offset zero is gone whole, index pair and all, rather than trimmed. `kafka-dump-log --files orders-0/00000000000000000000.log` has nothing left to open. `kafka-get-offsets --topic orders --time earliest` says where the log begins now, and `du -sh orders-0` says what the deletion gave back. Open at this step: step 4, Retention took a file, not rows
- The default is back, and a value that is refused t+7.600s The key was DELETED, not zeroed. `kafka-configs --entity-type topics --entity-name orders --alter --delete-config retention.ms` is the command that removed it, and `kafka-configs --describe --entity-type topics --entity-name orders` is back to the two keys it opened with, with `kafka-topics --describe --topic orders` agreeing - which in this simulator means no time-based deletion at all, and on a real broker would mean falling back to log.retention.ms, seven days by default. Deleting the override is not the same as setting the value to zero. One last thing worth pressing: `kafka-configs --entity-type topics --entity-name orders --alter --add-config retention.ms=abc` never reaches the cluster here, because this terminal validates the value per key before dispatching and names the type Kafka declares for it; the real kafka-configs sends it and the broker rejects it with an InvalidConfigurationException. Open at this step: step 5, The default is back, and a value that is refused
The takeaway
kafka-configs works one key at a time on one entity, and what it writes is a TOPIC-level value that overrides whatever the topic would otherwise inherit. Two things are worth carrying away. Deleting a config is not the same as setting it low: --delete-config removes the override and the key goes back to its default, which in this simulator means no time-based deletion at all and on a real broker means the cluster-wide log.retention.ms, seven days out of the box. Either way it is not immediate deletion, which is what setting the value to zero would ask for. And retention frees space by unlinking whole segment files, so the log start jumps to a segment boundary and the records below it are gone from disk together - which is also why a consumer that was lagging into that prefix gets an out-of-range error rather than a shorter answer. This terminal validates the value you type per key before anything is dispatched, against the same whitelist that lets the change show up on the canvas and survive a shared link; on a real cluster it is the broker that rejects a bad value.
Configs in this scenario
-
retention.How long a topic with cleanup.policy=delete keeps a log segment before it may be deleted (default 604800000, 7 days); -1 removes the time limit. How retention.ms worksms
Embed this scenario
Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.
<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=13.2.2" title="Kafka Simulator: Change a topic config live" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/change-a-config/">Set retention.ms on a live topic with kafka-configs</a></p>Further reading
More in Terminal ops
- Kafka from the terminal: hosts, ssh and the KRaft controller
- Diagnose consumer lag with kafka-consumer-groups --describe
- Kafka log segments on disk: leader vs follower
- Under-replicated vs under-min-ISR partitions in Kafka
- Reset Kafka consumer group offsets with --to-earliest
- Kafka partition reassignment: execute, throttle, verify
- Graceful broker shutdown with kafka-server-stop
- Kafka tools on two clusters: --bootstrap-server decides