KSQL select * from table returns no results and prompt is not responsive

Question

I'm using kafka-connect to stream rows from a MySQL table into a kafka topic. This is working great.

Then I create a table with:

CREATE TABLE mytable (id INT, email VARCHAR, gender VARCHAR, first_name VARCHAR, last_name VARCHAR) WITH (KAFKA_TOPIC='mysql-my-table', VALUE_FORMAT='AVRO', KEY='id');

This also works, as I can confirm by doing:

LIST TABLES; 
DESCRIBE EXTENDED mytable;

I see mytable .

The problem is when I execute

SELECT * FROM mytable;

Then I get no results and the prompt is unresponsive, I have to press ctrl+c to get control back.

What could be the problem?

Answer 1

After some time trying different things and reading the documentation I found the problem.

The field id for the messages in my topic are of type ? INT and according to KSQL TABLES documentation they need to be of type VARCHAR

So I followed the steps to fix this as described here and now everything is working fine.

So the steps are as follow:

-- Create a stream on the original topic
CREATE STREAM users_with_wrong_key_format (userid INT, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users', VALUE_FORMAT='JSON');

-- Derive a new stream with the required key changes.
-- 1) The CAST statement converts the key to the required format.
-- 2) The PARTITION BY clause re-partitions the stream based on the new, converted key.
CREATE STREAM users_with_proper_key
WITH(KAFKA_TOPIC='users-with-proper-key') AS
SELECT CAST(userid as VARCHAR) as userid_string, username, email
FROM users_with_wrong_key_format
PARTITION BY userid_string;

-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
        VALUE_FORMAT='JSON',
        KEY='userid_string');

-- Now you can create the table on the properly keyed stream.
CREATE TABLE users_table (userid_string VARCHAR, username VARCHAR, email VARCHAR)
WITH (KAFKA_TOPIC='users-with-proper-key',
    VALUE_FORMAT='JSON',
    KEY='userid_string');