Apache Kafka Notes
  • Introduction
  • Kafka Gurantees
  • Kafka Producers
    • Partitioning Strategy
    • Sending Messages
  • Intro
  • Topics
  • Messages
  • Partitions
  • Consumers
    • Group Coordinator
    • Notes
  • Zookeeper
  • HANDLING LARGE MESSAGES IN KAFKA
  • Version Incompatibilities
  • Use Cases
  • Message Delivery Semantics
  • API
  • References
  • Clusetr Setup
    • Single Node Single Borker
    • Single Node Multi Broker
    • Multi Node Mutil Broker
Powered by GitBook
On this page

Was this helpful?

  1. Consumers

Notes

PreviousGroup CoordinatorNextZookeeper

Last updated 5 years ago

Was this helpful?

The 0.9.0 consumer assumes all offsets are stored in Kafka Brokers ( they used to be stored in zookeeper prior 0.9 , In earlier Kafka releases (before 0.8.2), consumers commit their offsets to ZooKeeper.) . The way how it is done from 0.9 is , Kafka now provides an alternative to store consumer offsets into a special and separate Kafka topic, which is replicated and highly available.

If you have an existing old consumer, you’ll need to move your offsets to Kafka from ZooKeeper by setting

offsets.storage=kafka

and

dual.commit.enabled=true

in your 0.8.2 consumer code. Once you’ve moved the offsets to Kafka, you can start using a new consumer at the same place. Another option is to save out the current old consumer’s offset location and start the new consumer at the same offset.

http://www.jesse-anderson.com/2016/04/kafka-0-9-0-changes-for-developers/