[英]How to consume/read messages from KAFKA topic with filter condition
I have a topic with different messages. 我有一个主题不同的消息。 But I need to consume only certain messages irrespective of the order in which message published.
但是我只需要消耗某些消息,而不考虑消息发布的顺序。 I need to apply data filter condition on specific field/attribute of the message.
我需要在邮件的特定字段/属性上应用数据过滤条件。 like file-name element of the JSON message is "XXXX.txt".
JSON消息的文件名元素类似“ XXXX.txt”。 Simply say consume message with where condition like traditional DB.
只需说一下使用传统DB这样条件的消费消息。
.I need to use C# client
我需要使用C#客户端
Then your only option would be to subscribe and poll like normal, adding an if statement for your condition 然后,您唯一的选择就是像平常一样进行订阅和轮询,为您的条件添加if语句
Borrowing snippet from the Confluent consumer example 融合消费者示例中的借款摘要
// TODO: build consumer that reads string messages
consumer.Subscribe(topics);
...
while (true) {
var consumeResult = consumer.Consume(cancellationToken);
string value = consumeResult.Value;
// TODO: parse string value to JSON object using your favorite JSON library
// Add your condition
if (jsonObj["file-name"].Equals("XXXX.txt"))
{
Console.WriteLine($"Received message at {consumeResult.TopicPartitionOffset}: {value}");
}
Regarding the JSON parser, you can either make a Dictionary or a strongly typed class, but this example assumes a dictionary. 关于JSON解析器,您可以创建一个Dictionary或一个强类型的类,但是本示例假定使用字典。 How can I deserialize JSON to a simple Dictionary<string,string> in ASP.NET?
如何在ASP.NET中将JSON反序列化为简单的Dictionary <string,string>?
irrespective of the order in which message published.
不管消息发布的顺序如何。
As with any consumer, order is guaranteed within a partition 与任何消费者一样,可以保证分区内的订单
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.