[英]Kafka Producer Callback performance
I have Kafka Produce which sends the message to kafka.And i log the message in database in the both onsucess and onFailure with the help stored procedure.我有 Kafka Produce,它将消息发送到 kafka。我在帮助存储过程的 onsucess 和 onFailure 中将消息记录在数据库中。 As shown in the code i am using asynchronous
如代码所示,我正在使用异步
should i mark my callStoredProcedure method in the repository as synchronised to avoid deadlocks?我应该将存储库中的 callStoredProcedure 方法标记为同步以避免死锁吗? i believe synchronised is not needed as callback will be executed sequentially in a single thread.
我相信不需要同步,因为回调将在单个线程中顺序执行。
from the below link从下面的链接
https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html
Note that callbacks will generally execute in the I/O thread of the producer and so should be reasonably fast or they will delay the sending of messages from other threads.
请注意,回调通常会在生产者的 I/O 线程中执行,因此应该相当快,否则它们会延迟从其他线程发送消息。 If you want to execute blocking or computationally expensive callbacks it is recommended to use your own Executor in the callback body to parallelize processing.
如果您想执行阻塞或计算量大的回调,建议在回调主体中使用您自己的 Executor 来并行处理。
Should i execute callbacks in other thread?我应该在其他线程中执行回调吗? And can u share the code snippet how to excute callback in other thread.
你能分享一下如何在其他线程中执行回调的代码片段吗? like parallelise callback in 3 threads
像 3 个线程中的并行回调
My code snippet我的代码片段
@Autowired
private Myrepository myrepository;
public void sendMessageToKafka(List<String> message) {
for (String s : message) {
future = kafkaTemplate.send(topicName, message);
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
@Override
public void onSuccess(SendResult<String, String> result) {
System.out.println("Message Sent " + result.getRecordMetadata().timestamp());
myrepository.callStoredProcedure(result,"SUCCESS");
}
@Override
public void onFailure(Throwable ex) {
System.out.println(" sending failed ");
myrepository.callStoredProcedure(result,"FAILED");
}
});
}
private final ExecutorService exec = Executors.newSingleThreadExecutor();
...
this.exec.submit(() -> myrepository.callStoredProcedure(result,"SUCCESS"));
The tasks will still be run on a single thread (but not the Kafka IO thread).任务仍将在单个线程上运行(但不是 Kafka IO 线程)。
If it can't keep up with your publishing rate, you might need to use a different executor such as a cached thread pool executor or Spring's ThreadPoolTaskExecutor
.如果它跟不上您的发布速度,您可能需要使用不同的执行器,例如缓存线程池执行器或 Spring 的
ThreadPoolTaskExecutor
。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.