如何使用Java修改一个kafka主题的消息并发送到另一个kafka主题？

Question

I have created producer which produces MSG to one topic A and My need is I want to do change in that MSG and want to send it to another topic B, I'm trying to do this by Kafka streams,but not sure is it a right way or not. if it requires Kafka streams for that then please share what should be the code to write?

Answer 1

There are many options how you can to this.

Kafka's Streams API is one possibility with the advantage that you don't add a new dependency to your project. As simple stream.map(...).to(...) as program should do this. Check out the docs and examples for more details:

• http://docs.confluent.io/current/streams/index.html
• https://github.com/confluentinc/examples

You could even just use plain KafkaConsumer and KafkaProducer to get this done (but using Kafka's Streams API seem to be more natural than this).

Of course, you can also use a stream processing framework like Apache Flink, Apache Storm, Apache Apex, Heron (or maybe Apache Spark) -- for sure, there are others too. The program would be similarly simple to write as sketched above. But you need to setup a processing cluster to deploy you job for those stream processing framework.

Answer 2

You should look into using Apache storm , it will be very suitable for your needs.

I have used it and it does very well and gels like water in milk with kafka. It is also very easy to implement.

Best of luck