Kafka troubleshooting
Kafka transaction timed out (transaction.timeout.ms)
org.
The transaction stayed open longer than transaction.timeout.ms (default 60 seconds), so the transaction coordinator aborted it and bumped the producer epoch. There is no TRANSACTION_TIMED_OUT error code: the producer finds out on its next call, usually as InvalidProducerEpochException ("Producer attempted to produce with an old epoch"). Recreate the producer and keep transactions shorter than the timeout.
Reproduce it in the playground2 steps beforeCluster ready · txn timeout armed. t+0.000s
orders has 1 partition, RF=3. transaction.timeout.ms is 2000 ms. A transactional producer tx1 is idle; group rc is read_committed. Press ▶.
Open at this step: step 1, Cluster ready · txn timeout armed1 step beforeOpen · uncommitted, timer running. t+1.000s
tx1 began a transaction (the simulator schedules its txn_timeout for 2100, begin + 2000 ms; Kafka would count from the first partition added, at 200) and wrote offset 0. The record is replicated (HW advanced) but the LSO is held at 0 - read_committed sees nothing while the timer counts down.
Open at this step: step 2, Open · uncommitted, timer runningThe moment it happensTimed out · auto-aborted. t+2.600s
tx1 never committed. Once transaction.timeout.ms passed, the coordinator's periodic check auto-aborted it: the offset-0 record becomes an aborted range, the LSO releases to the high watermark, and tx1 is errored (the simulator calls it transaction_timed_out; Kafka bumps the epoch, so tx1's next send or commit fails with an epoch error). read_committed skips the aborted record entirely.
Open at this step: step 3, Timed out · auto-abortedWhat it means
The timer starts when the first partition is added to the transaction. Every transaction.abort.timed.out.transaction.cleanup.interval.ms (default 10 seconds) the coordinator looks for transactions past their timeout, bumps the producer epoch to fence the producer, and writes abort markers, logging "Completed rollback of ongoing transaction for transactionalId orders-tx due to timeout", so read_committed consumers skip the records and the last stable offset moves on. KIP-588 proposed a dedicated TRANSACTION_TIMED_OUT error, but Kafka 4.x has none, so the producer learns about the abort indirectly. A send() in the aborted transaction fails with INVALID_PRODUCER_EPOCH (error code 47), and commitTransaction() then throws that InvalidProducerEpochException: "Producer with transactionalId 'orders-tx' and (producerId=1000, epoch=3) attempted to produce with an old epoch". A producer that sends nothing more and calls commitTransaction() directly is rejected by the coordinator instead: with ProducerFencedException on the original transaction protocol (transaction.version=1), with InvalidTxnStateException on the KIP-890 protocol (transaction.version=2), which Kafka 4.0 brokers enable by default and which also needs 4.0 or newer clients. Both are fatal for that producer. The simulator labels the error TransactionTimedOut; in scenario 04.2.4 the coordinator aborts an abandoned transaction and releases the LSO, which is the broker side of what this page describes.
Likely causes, most common first
- Work inside the transaction takes longer than the timeout. Processing a large poll batch, calling a slow external service, or waiting on a lock between beginTransaction() and commitTransaction() pushes the transaction past transaction.timeout.ms.
- The producer stalled. A long GC pause, a frozen or CPU-throttled container, or a network partition between the producer and the coordinator. KIP-588 names short client inactivity from a network partition or a long GC as the usual trigger.
- A transaction was left open. A code path calls beginTransaction(), then returns or throws without commitTransaction() or abortTransaction(), so nothing ends the transaction until the coordinator does. Scenario 04.2.4 reproduces this: the producer never resolves its transaction and the coordinator aborts it at the timeout.
How to fix it
- Catch the exception, close the producer, create a new one with the same transactional.id and call initTransactions(); then rewind the consumer to its last committed offsets, or let it rebalance, and redo the work, since none of the aborted records or offsets were committed.
- Keep transactions short: commit after a bounded number of records or a bounded time, and lower max.poll.records if one poll batch takes too long to process.
- Raise transaction.timeout.ms when transactions are legitimately long. It has to stay at or below the broker's transaction.max.timeout.ms (default 15 minutes) or initTransactions() fails with InvalidTxnTimeoutException, and a longer timeout lets an abandoned transaction hold back read_committed consumers for longer.
- Check what the coordinator holds with kafka-transactions.sh --bootstrap-server localhost:9092 describe --transactional-id my-txn-id: it shows the TransactionState, the ProducerEpoch, TransactionTimeoutMs and how long the current transaction has been open.
- Find stalls in GC logs and in your own measurement of the time between beginTransaction() and commitTransaction().
What not to do
- Confusing this with the TimeoutException that commitTransaction() throws when max.block.ms runs out: that one means the acknowledgement did not arrive in time, the commit may still complete, and the Javadoc allows only retrying the commit or closing the producer.
Does the client retry it?
No. The transaction is already aborted and its records will never be committed. Kafka 4.x classifies InvalidProducerEpochException as application-recoverable: close the producer, create a new one with the same transactional.id and redo the work of the aborted transaction. The exception's own Javadoc still describes recovering with abortTransaction(); the 4.x guidance (KIP-1050) is to restart the producer.
Configs involved
transaction.The longest a transaction may stay open, counted from the first partition added to it, before the transaction coordinator aborts it (default 60000, 1 minute); it may not exceed the broker transaction.max.timeout.ms.timeout. ms transaction.The largest transaction.timeout.ms a producer may ask for (default 900000, 15 minutes); a larger value fails initTransactions() with InvalidTxnTimeoutException.max. timeout. ms transaction.How often the transaction coordinator looks for transactions past their transaction.timeout.ms and aborts them (default 10000, 10 seconds), so an abort can come up to this much after the timeout.abort. timed. out. transaction. cleanup. interval. ms transactional.A stable name that turns on transactions for a producer (and implies idempotence); a new producer that calls initTransactions() with the same id fences the older one.id isolation.read_uncommitted (the default) returns every record, including those of open and aborted transactions; read_committed returns only committed transactional records and stops at the last stable offset.level max.The most records one poll() returns (default 500); a smaller value shortens the processing between two polls, which keeps a group member inside max.poll.interval.ms.poll. records max.How long send(), partitionsFor() and the transaction methods may block, waiting for metadata, for room in buffer.memory or for the transaction coordinator, before they throw a TimeoutException (default 60000, 1 minute).block. ms
Reproduce it in the simulator
- Kafka transaction.timeout.ms: the coordinator aborts (Transaction timeout) Open at the moment it happens: Kafka transaction.timeout.ms: the coordinator aborts
Sources
- Apache Kafka 4.3: producer config transaction.timeout.ms
- Apache Kafka 4.3: broker config transaction.abort.timed.out.transaction.cleanup.interval.ms
- Apache Kafka 4.3: broker config transaction.max.timeout.ms
- Apache Kafka 4.3 Javadoc: InvalidProducerEpochException
- Apache Kafka 4.3 Javadoc: KafkaProducer (commitTransaction)
- KIP-588: Allow producers to recover gracefully from transaction timeouts
- Apache Kafka 4.3: transaction protocol (KIP-890, transaction.version)
- KIP-1050: Consistent error handling for Transactions (InvalidProducerEpochException is application-recoverable)
- Apache Kafka 4.3: using transactions (exception categories, recovery)
- Apache Kafka 4.3.0 source: TransactionCoordinator.scala (abortTimedOutTransactions, EndTxn checks)
- Apache Kafka 4.3.0 source: TransactionsTest.scala (testFencingOnTransactionExpiration)
- Apache Kafka 4.3.0 source: Errors.java (INVALID_PRODUCER_EPOCH, code 47)
- Apache Kafka 4.3.0 source: TransactionsCommand.java (kafka-transactions.sh describe)
- Apache Kafka 4.3.0 source: TransactionManager.java (attempted to produce with an old epoch)