Where KSQL stream topic is stored?

Question

Running command:

`λ oc exec -it ms-ksql-db-server-deployment-76f454b75-mjfbh bash`

[appuser@ms-ksql-db-server-deployment-76f454b75-mjfbh ~]$ `ksql`
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

              ===========================================
              =       _              _ ____  ____       =
              =      | | _____  __ _| |  _ \| __ )      =
              =      | |/ / __|/ _` | | | | |  _ \      =
              =      |   <\__ \ (_| | | |_| | |_) |     =
              =      |_|\_\___/\__, |_|____/|____/      =
              =                   |_|                   =
              =        The Database purpose-built       =
              =        for stream processing apps       =
              ===========================================

Copyright 2017-2022 Confluent Inc.

CLI v0.26.0, Server v0.26.0 located at http://localhost:8088
Server Status: RUNNING

Having trouble? Type 'help' (case-insensitive) for a rundown of how things work!

ksql> CREATE STREAM CLIENT_STREAM WITH (KAFKA_TOPIC='private-import-clients',VALUE_FORMAT='AVRO');

 Message
----------------
 Stream created
----------------
ksql> SET 'auto.offset.reset'='earliest';
Successfully changed local property 'auto.offset.reset' to 'earliest'. Use the UNSET command to revert your change.
ksql> SHOW STREAMS;

 Stream Name         | Kafka Topic                 | Key Format | Value Format | Windowed
------------------------------------------------------------------------------------------
 CLIENT_STREAM       | private-import-clients      | KAFKA      | AVRO         | false
 KSQL_PROCESSING_LOG | default_ksql_processing_log | KAFKA      | JSON         | false
------------------------------------------------------------------------------------------
ksql> SELECT * FROM CLIENT_STREAM EMIT CHANGES;

I am writing KSQL Streams from a topic private-import-clients which i assume that the result of my Stream created will give origin a topic CLIENT_STREAM or some other name. Is it correct? If so, where does KSQL stores that topic CLIENT_STREAM ? I have a Spring boot app whcih will consume the topic CLIENT_STREAM .

Answer 1

CREATE STREAM statement is a DDL statement, so the action is to update the ksqlDB metadata.

Each ksqlDB Server has an internal, in-memory metadata store, or metastore, that it builds as it receives DDL statements. The metastore is an in-memory map. For each new DDL statement, the ksqlDB engine adds an entry to the metastore.

The ksqlDB engine translates the DML statement into a Kafka Streams application

This page explains how KSQL creates logical, physical plans and how it handles queries.

So it stores only local meta data in a in-memory map for DDL statements and behaves similar to Streams for DML statements.