我可以在Logstash和Elasticsearch之间使用Kafka吗？ （使用两个卡夫卡）
[英]Can I use Kafka between Logstash and Elasticsearch? ( Using two Kafka)
问题描述
I'm trying to integrate Apache Kafka with Elastic Stack (Beats, Logstash, Elasticsearch, and Kibana) 
我正在尝试将Apache Kafka与Elastic Stack （Beats，Logstash，Elasticsearch和Kibana）集成在一起
From the diagram, Kafka is located between Beats and Logstash. 从图中可以看出，Kafka位于Beats和Logstash之间。 I was wondering if I can put another Kafka between Logstash and Elasticsearch. 我想知道是否可以在Logstash和Elasticsearch之间放置另一个Kafka。 (Where I drew with a red pen.) （我用红笔画的地方。）
Two Kafka sound okay? 两个卡夫卡的声音还好吗？
Any ideas or thoughts to share? 有什么想法或想法要分享吗？
1 个解决方案
解决方案1
1 已采纳 2019-05-13 07:53:06
Yes. 是。
Logstash can write to Kafka as an output.
Logstash可以作为输出写入Kafka。 You can use Kafka Connect Elasticsearch for streaming from Kafka into Elasticsearch.
您可以使用Kafka Connect Elasticsearch从Kafka流到Elasticsearch。
If you want to buffer/scale the output from Logstash by using Kafka here, it is possible and would make sense. 如果您想在此处使用Kafka缓冲/缩放Logstash的输出，则可能并且很有意义。
But bear in mind that you could also: 但是请记住，您还可以：
(a) write from Beats to Kafka and do any processing with KSQL/Kafka Streams etc to write back to Kafka and then Kafka Connect to Elasticsearch or (b) just write from Logstash to Elasticsearch （a）从Beats写入Kafka，并使用KSQL / Kafka Streams等进行任何处理，然后再写入Kafka，然后再将Kafka连接到Elasticsearch，或者（b）从Logstash写入Elasticsearch
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.