How to use acknowledgement when reading messages from kafka topic in confluent kafka go?

Question

I am developing a push notification that sends many messages to clients. Messages are published into topics and Subcribers read message from the same topic. In case of errors right after reading message from topic offset is incremented and even though I could not send message my Subscriber needs to read the next message and send it. By errors I mean the server being down or something serious.

How can I read messages with acknowledgements?

Answer 1

I am not sure I undestand what you mean by

In case of errors right after reading message from topic offset is incremented and even though I could not send message my Subscriber needs to read the next message and send it

What I understood is that you want to manage how your consumers handle acknowledgements (commits to the _consumer_offsets).

So Kafka allows consumers to track their position (offset) in each partition by producing a message to Kafka in the __consumer_offsets topic.

3 options are available :

1. Auto commit : with enable.auto.commit=true , commits automatically every auto.commit.interval.ms (default 5s).
2. Synchronous commit : commit synchronously using commitSync() explicitly which commits the latest offset returned by poll() and retries if failure until its gets a confirmation.
3. Asynchronous Commit : Previous approach waits until the broker responds ackowledges the commit, it makes things slower. We can use commitAsync() which is not blocking and will not retry if it fails. It is faster. We can pass a callback to commitAsync().

So basically, you can let commits be handled automatically. Commit synchronously and wait for the broker's ack or commit asynchronously with a callback.

Hope this answers your question.

Best regards.