--- title: "Kafka auto.offset.reset: earliest vs latest" description: "What Kafka auto.offset.reset does, earliest vs latest vs none, when it applies and when it does not, shown step by step in a live simulator." url: https://monedula.dev/kafka-simulator/configs/auto-offset-reset/ --- # auto.offset.reset: earliest vs latest Consumer config · default latest · Apache Kafka 4.3. auto.offset.reset decides where a consumer starts reading when its group has no committed offset for a partition, or the committed offset no longer exists in the log. latest (the default) starts at the end and reads only records produced from then on; earliest starts at the oldest record still retained and reads the whole backlog. It does nothing for a group that already has a valid committed offset. ## At a glance - Config: `auto.offset.reset` (consumer) - Default: latest - Valid values: latest, earliest, by_duration: (such as by_duration:PT1H), none - Reference: [Apache Kafka documentation](https://kafka.apache.org/43/configuration/consumer-configs/#consumerconfigs_auto.offset.reset) ## Values - `latest`: Start at the log end offset. Records already in the partition are skipped; the consumer sees only what is produced after it was assigned the partition. - `earliest`: Start at the log start offset, the oldest record retention has not deleted yet, and read everything from there. - `by_duration:`: Start at the first record with a timestamp at most that long ago, such as by_duration:P1D for the last day. Added in Kafka 4.0 (KIP-1106). - `none`: Pick nothing: poll() throws NoOffsetForPartitionException when there is no committed offset, or OffsetOutOfRangeException when it is out of range, and the application decides. ## What it does A consumer group resumes each partition from its committed offset. auto.offset.reset is only the fallback for the two cases where that offset is missing or useless: a group that has never committed for the partition (a new group, a newly added partition, or a group whose offsets expired after offsets.retention.minutes, 7 days by default, with no members), and a committed offset that retention has already deleted, which the broker answers with OFFSET_OUT_OF_RANGE. Restarting a consumer, deploying a new version or rebalancing does not apply it as long as the group still has a valid committed offset, so changing the setting on an existing group usually changes nothing. ## The trade-off earliest never skips data, but a new group reads the whole retained log first, which can be days of records to process before it catches up, and a group recovering from an out-of-range offset reprocesses whatever is left. latest starts instantly and never replays, but it silently skips everything produced before the group first joined, and when partitions are added to a topic, records producers write to a new partition before the group is assigned it are skipped too. none makes the choice explicit at the cost of handling the exception yourself. ## Advice - Use earliest for a group that must see every record, such as one that builds a table or a search index from the topic; latest suits groups that only care about new events, such as live alerting. - To move an existing group, reset its offsets with kafka-consumer-groups.sh --reset-offsets while the group is stopped; changing auto.offset.reset does not move a group that has committed offsets. - Treat an OFFSET_OUT_OF_RANGE reset as a symptom: the group fell behind retention.ms and lost records it never read. Alert on consumer lag before it happens. ## See it in the simulator - [Kafka auto.offset.reset: earliest vs latest vs none](https://monedula.dev/kafka-simulator/scenarios/auto-offset-reset/) (auto.offset.reset), step 2: g-latest starts at the END · g-earliest at the START. Both groups just joined. g-latest (reset=latest) seeked to the END - position 3, so it will read none of O1-O3. g-earliest (reset=earliest) seeked to the START - position 0, about to replay all three. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=02.1.2&t=1) - [Kafka retention.ms and a consumer that falls behind](https://monedula.dev/kafka-simulator/scenarios/time-retention/) (Time-based retention (retention.ms)), step 4: OFFSET_OUT_OF_RANGE · reset to earliest. On its next poll the group fetched from offset 0, but the log now starts at 2. auto.offset.reset=earliest seeks it to logStartOffset (2), it reads the surviving 2-3 and advances to HW=4. A lagging consumer recovered by jumping past the deleted prefix. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=05.1.1&t=3) - [Kafka OFFSET_OUT_OF_RANGE with auto.offset.reset=none](https://monedula.dev/kafka-simulator/scenarios/offset-out-of-range/) (Offset out of range), step 3: OFFSET_OUT_OF_RANGE · group stuck. The group restarted and polled from offset 0, but the log now starts at 3. The broker answers OFFSET_OUT_OF_RANGE, and with auto.offset.reset=none the consumer has no fallback, so poll() throws OffsetOutOfRangeException and the group is stuck. Set auto.offset.reset to earliest or latest to recover. [Open at this step](https://monedula.dev/kafka-simulator/playground/?scenario=05.1.3&t=2) ## Related configs - [Kafka retention.ms: time-based retention explained](https://monedula.dev/kafka-simulator/configs/retention-ms/) - [Kafka max.poll.interval.ms: slow processing and rebalances](https://monedula.dev/kafka-simulator/configs/max-poll-interval-ms/) - [Kafka read_committed vs read_uncommitted (isolation.level)](https://monedula.dev/kafka-simulator/configs/isolation-level/)