cannot get data from table KSQL

Question

I create a rekeyed stream

CREATE STREAM details_stream_rekeyed2 as \
  select CONCAT(IdSeq,IdTime,'') as root,ServerId,Server,\
         IdTime ,IdSeq \
  from voip_details_stream  \
  partition by root;

select from this stream and i get > 100 items

then i try to create a table

create table voip_details_table3 \
  (ROOT varchar,ServerId long , Server varchar ,IdTime varchar,IdSeq long ) \
  with ( kafka_topic = 'DETAILS_STREAM_REKEYED2', \
         value_format = 'json',\
         key='ROOT');

when i run

SELECT ROWKEY,ROOT  FROM VOIP_DETAILS_TABLE3;

i get the only less than 10 items likes ;

12018-04-04T18:56:35.080-04:00 | 12018-04-04T18:56:35.080-04:00

i run the command:

kafkacat -C -K: -b "$BROKER_LIST" -f 'Key:    %k\nKey Bytes: %K\nValue:  %s\nValue Bytes: %S\n\n' -t DETAILS_STREAM_REKEYED2

everething is ok. and i get data likes

 Key:    12018-02-05T15:16:07.113-05:00
 Key Bytes: 30
 Value:  {"SERVER":null,"IDSEQ":1,"ROOT":"12018-02-05T15:16:07.113-05:00","SERVERID":null,"SESSIONIDTIME":"2018-02-05T15:16:07.113-05:00"}

Value Bytes: 158

where the problem ?

Answer 1

A KSQL table differs from a KSQL Stream, in that it gives you the latest value for a given key . So if you are expecting to see the same number of messages in your table as your source stream, you should have the same number of unique keys.

If you're seeing fewer messages then it suggests that ROOT is not unique.

Depending on the problem that you're modelling, you should either :

• (a) be using a Stream not a Table, or
• (b) change the key that you are using

Ref:

• https://docs.confluent.io/current/streams/concepts.html#duality-of-streams-and-tables
• http://www.michael-noll.com/blog/2018/04/05/of-stream-and-tables-in-kafka-and-stream-processing-part1/