Unable to receive data on the Lenses Kafka through the MQTT Broker

Question

I have a particular problam regarding the use of the Lenses Kafka. Sensor data is not received on its end.

Namely, I try to create a Network connection using the Docker container, where each container represents on service. The services are following:

1. The first is the sensor which sends data using the MQTT protocol (in this case a Raspberry Pi with sensor attached)
2. MQTT Broker - Mosquitto in this case, used to get messages and interact with Lenses
3. Lenses Kafka - system used for interaction between Mosquitto in this case and other services (like InfluxDB on the other end)

I can say for certain that the data that is sent is successfuly received, since I use the Apache JMeter to check on the received data from the Mosquitto site. The problem arrises on the Lenses side, where the connection is recognized, but not data is received and the following error is showed:

MqttException (0) - java.net.NoRouteToHostException: Host is unreachable (Host unreachable) 
 at org.eclipse.paho.client.mqttv3.internal.ExceptionHelper.createMqttException(ExceptionHelper.java:38)
 at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:715)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
 at java.util.concurrent.FutureTask.run(FutureTask.java:266)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748) 
Caused by: java.net.NoRouteToHostException: Host is unreachable (Host unreachable)
 at java.net.PlainSocketImpl.socketConnect(Native Method)
 at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
 at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
 at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) 
 at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
 at java.net.Socket.connect(Socket.java:589) 
 at org.eclipse.paho.client.mqttv3.internal.TCPNetworkModule.start(TCPNetworkModule.java:84)
 at org.eclipse.paho.client.mqttv3.internal.ClientComms$ConnectBG.run(ClientComms.java:701) 
... 7 more

The configuration code that was used on the Lenses side to read data is:

connector.class=com.datamountaineer.streamreactor.connect.mqtt.source.MqttSourceConnector
connect.mqtt.clean=true
connect.mqtt.timeout=1000
tasks.max=1
connect.mqtt.keep.alive=1000
name=Mosquitto
connect.mqtt.kcql=INSERT INTO kafka_sensor_data SELECT * FROM /sensor_data WITHCONVERTER=`com.datamountaineer.streamreactor.connect.converters.source.JsonSimpleConverter`
connect.mqtt.service.quality=1
connect.mqtt.client.id=dm_source_id
connect.mqtt.converter.throw.on.error=true
connect.mqtt.hosts=tcp://XXX:1883

(XXX is only for security purposes, there is already a tested IP address).

This approach has actually worked until recently, but all of the sudden I am becoming this error now. I checked the connections and the port connectivity, but everything there is working. Not really sure where to check anymore.

Answer 1

After some time I managed to find the issue.

It was by using the docker container. While the service managed to keep the ports and addresses open and organised to the outside network, there were issues from the inside, as Lenses could not recognize the MQTT Broker.

After configuring my own network rather than letting it be created on the default I managed to fix this.