How to process stream data when KSQL server running in Headless mode?

Question

I have to deploy KSQL on production server so I have changed interactive mode to headless mode. Created a SQL script to load all streams at container startup. Now I want to see stream data. How many ways are there to see those live streams like we can connect KSQL server to control center and achieve this. But I don't want to make a dependency like we need to buy enterprise edition of Confluent. Please give your suggestions or inputs.

Answer 1

KSQL (or ksqlDB to be technically correct) is an streaming analytics layer built on top of Apache Kafka. While this is mouthful and could means different things for different people -- in your particularly case it means that if you need to see the streams of data you can simply monitor the Kafka topics created by the streams/tables of your SQL script.

in ksqlDB, every stream and/or table that is backed by a continuous query flushes its results into a generated topic that contains a prefix (this is dictated by the ksql.output.topic.name.prefix configuration parameter) and the name of the stream/table. Just use any CLI tool (such as kafka-console-consumer, kafkacat, etc) to connect to the Kafka cluster and print the topic content.