简体   繁体   中英

Kafka topic returns 0 messages

I have a real-time process that streams events occuring in MongoDB directly to Snowflake using Confluent Platform. Confluent is established on my local machine as a docker image and I see no issues:

     Name                    Command                  State                         Ports
------------------------------------------------------------------------------------------------------------
broker            /etc/confluent/docker/run        Up             0.0.0.0:9092->9092/tcp
connect           /etc/confluent/docker/run        Up (healthy)   0.0.0.0:8083->8083/tcp, 9092/tcp
control-center    /etc/confluent/docker/run        Up             0.0.0.0:9021->9021/tcp
ksql-datagen      bash -c echo Waiting for K ...   Up
ksqldb-cli        /bin/sh                          Up
ksqldb-server     /etc/confluent/docker/run        Up (healthy)   0.0.0.0:8088->8088/tcp
rest-proxy        /etc/confluent/docker/run        Up             0.0.0.0:8082->8082/tcp
schema-registry   /etc/confluent/docker/run        Up             0.0.0.0:8081->8081/tcp
zookeeper         /etc/confluent/docker/run        Up             0.0.0.0:2181->2181/tcp, 2888/tcp, 3888/tcp

Everything was working over last days and today I re-ran my process with powershell:

docker-compose exec broker bash and kafka-console-consumer --topic <MY_TOPIC> --bootstrap-server broker:9092

I noticed that when I insert data to Mongo's collection Kafka topic doesn't see it, instead returns Processed a total of 0 messages

How to debug what is going on? Connection passwords, topics are all set correctly and I don't know where to start debugging.

I used to get below result but now it's idle: 在此处输入图像描述

Ok, in Confluent Control Center Connect tab check setting for source connector.

For my mongo's connection, new URI syntax fixed the issue (the one where you don't point database, that is done in seperate tab)

在此处输入图像描述

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM