I’m writing this at 2:30 AM after six hours chasing a bottleneck in my integration tests. While implementing the Kafka adapter for @anabranch/eventlog, I hit a consistent 5-second delay on every admin.setOffsets call used for manual cursor commits, in tests where the consumer group was idle.

The name suggests a lightweight metadata call. It isn't. KafkaJS spins up a transitive consumer that joins the group, fetches to synchronize state, and then performs the update, and that internal consumer uses the default maxWaitTimeInMs of 5000ms:

// kafkajs/src/admin/index.js
const consumer = createConsumer({
  logger: rootLogger.namespace('Admin', LEVELS.NOTHING),
  cluster,
  groupId,
})

// Which eventually calls createConsumer with a hardcoded default:
({
  // ...
  maxWaitTimeInMs = 5000,
  // ...
}) => {
  // ...
}

With an idle group or a topic with no new data, there is nothing to return, so the consumer blocks for the full maxWaitTimeInMs before the fetch times out and the operation continues. A sane default for a production consumer; pure dead time for an administrative update.

Fitting Kafka into a Clean Interface

@anabranch/eventlog defines commitCursor, which reads like a straightforward metadata update. Kafka's offsets aren't a key-value store, though: they're coupled to consumer group coordination, so committing out-of-band has to account for a group that might be rebalancing or empty.

A Configurable Mimic

The fix is to do what KafkaJS does internally, but own the timing. Instead of admin.setOffsets, spin up a temporary consumer with a much lower maxWaitTimeInMs (100ms).

A single commitCursor task then:

  1. Initialize a temporary consumer with a configurable dummyMaxWaitTimeInMs.
  2. Connect and subscribe to the relevant topic.
  3. Use seek to point the consumer to the exact target offsets for each partition.
  4. Perform a standard commitOffsets and immediately disconnect.

Owning the lifecycle and timing of the transitive consumer keeps the group offset commit correct and drops the latency from 5000ms to ~150ms per call. It’s still not fantastic, it remains the slowest integration test in the suite, but Kafka is bulky by nature, and 150ms is a far better baseline than five seconds.

The source for this adapter and the rest of the Anabranch monorepo is available on GitHub and JSR.