在 KSQLDB 中从具有字符串类型值的主题创建表

Question

How can one create a table from a topic which contains value of type String?

We have some topics that contains rdf data embedded inside strings, in a sense it is just a string value. Based on the KSQLDB documentation, we need to use value_format='KAFKA' with WRAP_SINGLE_VALUE=false given that it is an anonymous value.

CREATE SOURCE Table source_table_proxy (
    key VARCHAR primary KEY,
    value VARCHAR
) WITH (
    KEY_FORMAT='KAFKA',
    VALUE_FORMAT='KAFKA', 
    WRAP_SINGLE_VALUE=false,
    kafka_topic = 'topic'
); 

This is the topic info:

Key Type: STRING
Value Type: STRING
Topic Info
Partitions: 12
Replication: 1

Weirdly we get the following error:

The 'KAFKA' format only supports a single field. Got: [`VALUE` STRING, `ROWPARTITION` INTEGER, `ROWOFFSET` BIGINT]

Is there any workaround this issue?

Answer 1

Unclear why you need KAFKA format.

The JSON format will work for plain (primitive) strings as well

supports reading and writing top-level primitives, arrays and maps.

For example, given a SQL statement with only a single field in the value schema and the WRAP_SINGLE_VALUE property set to false:

CREATE STREAM x (ID BIGINT) WITH (VALUE_FORMAT='JSON', WRAP_SINGLE_VALUE=false,...);

And a JSON value of:

 10

ksqlDB can deserialize the values into the ID field of the stream

https://docs.ksqldb.io/en/latest/reference/serialization/#top-level-primitives-arrays-and-maps