将事件发送到特定的Kafka主题分区，并通过flume hive sink读取
[英]send event to specific kafka topic partition and read it by flume hive sink
问题描述
I am using Kafka 0.10 and Flume 1.8. 
我正在使用Kafka 0.10和Flume 1.8。 I am trying to get information on below (but could not get it). 我正在尝试获取以下信息（但无法获取）。 So can any body please help me. 所以任何人都可以帮助我。
- Is there any way to send events to particular kafka topic partition 有什么方法可以将事件发送到特定的Kafka主题分区
- And if so, then can we read such events (coming to specific partition) with flume using hive sink 如果是这样，那么我们可以使用蜂巢接收器读取带有水槽的此类事件（进入特定分区）吗？
1 个解决方案
解决方案1
0 2017-11-16 16:39:10
I'm not sure I understand your motive... I'm pretty sure you can create a kafka topic with a single partition if you wish to. 我不确定我是否了解您的动机...如果您愿意，我很确定您可以使用单个分区创建kafka主题。
By doing this, you would know which partition and topic you were reading from. 这样，您将知道要从哪个分区和主题中读取。 It is also possible to have multiple sources in flume, so if you wish for a single service to read from multiple topics but for each topic to only have a single partition, you can easily do this. 槽中也可能有多个源，因此，如果您希望单个服务从多个主题中读取，但是对于每个主题只有一个分区，则可以轻松地做到这一点。
Apologies, I would have written this as a comment as it should really be a comment but I don't yet have that privilege in stackoverflow. 抱歉，我应该把它写为评论，因为它实际上应该是评论，但是我在stackoverflow中还没有特权。 Anyway, I hope this helps. 无论如何，我希望这会有所帮助。
声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.