Scenario 00.1.3 · Keys and partitioning

Kafka message keys: how a key picks the partition

Anatomy · Beginner · about 2 min · Apache Kafka 4.3

A record’s key decides its partition: same key → same partition (ordering), different keys spread, no key → the sticky partitioner spreads them over time.

What you start with

The record key determines its partition: Kafka hashes the key and maps it to a partition, so identical keys always land together while different keys spread out. With NO key there is nothing to hash, so the default (sticky) partitioner keeps keyless records on one partition for a while, then moves on - they still fan out across partitions over time. Watch keyed records pin by key, then watch keyless records spread (the simulator switches partition per batch so the spread is visible).

Step by step

1. Empty topic · 3 partitions t+0.000s A 3-partition topic. The producer uses the default partitioner: with a key, partition = hash(key) mod partitionCount; with no key, the sticky partitioner picks a partition per batch. Open at this step: step 1, Empty topic · 3 partitions
2. Different keys spread across partitions t+1.050s Keys A, C, F hash to different partitions, so the three keyed records spread out - one per partition at offset 0. Open at this step: step 2, Different keys spread across partitions
3. Same key → same partition t+1.450s A second record with key A lands in the very same partition as the first A, now at offset 1. The same key always maps to the same partition - that is what guarantees per-key ordering and co-partitioning. Open at this step: step 3, Same key → same partition
4. No key → a partition is chosen for the batch t+1.850s Now a record with NO key. There is nothing to hash, so the default (sticky) partitioner simply picks a partition for the batch - this keyless record lands on partition 0. Open at this step: step 4, No key → a partition is chosen for the batch
5. Keyless records spread across partitions t+2.350s Two more keyless records follow, each its own batch. The simulator moves the sticky partition as each batch closes, so the three keyless records spread across p0, p1, p2. Real Kafka 4.x stays on one partition until batch.size bytes (16 KB) have been sent, then switches to a random, load-weighted partition. Open at this step: step 5, Keyless records spread across partitions

The takeaway

Use a key when you need per-key ordering - every record for that key shares one partition. Leave the key empty for an even spread: the modern default is the sticky partitioner (it stays on one partition for about batch.size bytes before switching, rather than the older strict per-record round-robin), but either way keyless records distribute across all partitions over time.

Configs in this scenario

• batch.size The most bytes the producer puts in one batch for a partition (default 16384); a batch goes out when it is full or when linger.ms runs out, whichever comes first. How batch.size works

Embed this scenario

Paste this into a blog post, a course page or your team's docs. The scenario plays inside your page, and the link under it points readers to this walkthrough.

<iframe src="https://monedula.dev/kafka-simulator/embed/?scenario=00.1.3" title="Kafka Simulator: Keys and partitioning" width="100%" height="520" style="border:1px solid #d0d0d0;border-radius:8px" loading="lazy"></iframe>
<p><a href="https://monedula.dev/kafka-simulator/scenarios/keys-and-partitioning/">Kafka message keys: how a key picks the partition</a></p>

Further reading

Apache Kafka documentation

More in Anatomy

• Kafka topics, partitions and offsets explained
• Kafka write path: one record from producer to consumer
• Kafka offsets are per partition, not global
• Kafka replication factor, leader and follower replicas
• Kafka high watermark vs log end offset (LEO)
• Why Kafka consumers cannot read past the high watermark
• Kafka log start offset: why the earliest offset is not 0

All scenarios