Speeding up Message Hub Kafka Java Console sample

Question

I've been working with the Message Hub sample code found at this link: https://github.com/ibm-messaging/message-hub-samples

In particular, I've been trying to increase the throughput of the producer with the Kafka Java console example. I noticed the documentation in this snippet of code:

// Synchronously wait for a response from Message Hub / Kafka on every message produced. // For high throughput the future should be handled asynchronously. RecordMetadata recordMetadata = future.get(5000, TimeUnit.MILLISECONDS); producedMessages++;

I've already turned off the thread sleep found later in the code which also helped increase the throughput, but I was hoping I could get some help on implementing the future asynchronously in this block. Thanks in advance!

Answer 1

you have two basic options for handling the outcome of a produce request asynchronously

1) use the overloaded send with a completion callback argument, which will be invoked asynchronously: public Future<RecordMetadata> send(ProducerRecord<K, V> record, Callback callback);

if using the callback you may ignore the future.

2) pass the Future to some other thread you have created, and have it inspect the future for completion, while leaving the thread that calls send free to carry on.