高效的 MQTT 代理到 Kafka 代理的桥接

Question

I need to stream incoming sensor data from Mqtt broker to Kafka broker. What are the most efficient tools to do this ?

There are lots of brokers and some of them provide almost the same features.

Is there any broker that provides Kafka bridge plugin in an efficient way ?

Answer 1

There is an MQTT Kafka Connect connector for that.

https://github.com/evokly/kafka-connect-mqtt http://docs.datamountaineer.com/en/latest/mqtt.html

I have never used it but maybe it's worth to give it a try. let me know ;)

Answer 2

As alternative to Kafka Connect and a MQTT connector, you could also use the Confluent MQTT Proxy for this. If your only reason to have a MQTT Broker, then this is the much easier and more lightweight setup because you do not need a MQTT Broker this way. MQTT Proxy works the same way like Kafka' REST Proxy.

See this example of how to combine MQTT with Apache Kafka . Or go directly to the Github code: " Deep Learning UDF for KSQL for Streaming Anomaly Detection of MQTT IoT Sensor Data ".