如何在 Grafana 中可视化 Kafka 主题消息？

Question

Edited:

I've successfully configured the Kafka data source into Grafana, but how do I display the messages of my 'temp-sensor' topic in a dashboard? Those are temperature value output from the ds18x20 sensor as shown in the figure below from the Kafka console consumer.

Kafka console consumer

The topic is able to consume all the temperature values when the sensor is running, however, when I try to visualize the temperature data in Grafana, the time-series chart is not showing any value accordingly when I've already done the Kafka configuration.

Kafka dashboard

This is what I did -

Kafka data source / Kafka topics describe

Kafka-python lib in python3 -

import socket
import json
from kafka import KafkaProducer
producer = KafkaProducer(value_serializer=lambda m: json.dumps(m).encode('ascii'), bootstrap_servers='kafka.develop.com.my:9092')
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("", 1024))
s.listen(5)
while True:
    clt, adr=s.accept()
    print(f"Connection to {adr} established")
    while True:
        data = clt.recv(1024)
        f = float(data)
        print('Received', data)
        temp = {"temp":f}
        producer.send('temp-sensor', value=temp)
    clt.close()

Also, I've added Prometheus data source and imported Kafka Overview dashboard into Grafana. The dashboard shows that there are messages/bytes received from the 'temp-sensor' topic when I run to code.

Kafka Overview - Prometheus

So, I'm wondering what I have missed out or done wrong that my Kafka data source isn't displaying messages in the chart.

Answer 1

Unclear what DataSource you are using, but this one under Known Limitations says it only accepts JSON, not single floating point values, so update your Python code to use value=json.dumps({"value1": data})

They even have their own Python producer example .