如何在Ksql Server上分配查询

Question

thinking about scaling, i wonder how queries are distributed among KSQL Servers ?

1 - Let say you have the cli or confluent control center, does the cli connect to all server or one of them ? Can it send the query to multiple KSQL SERVER eg those with the same id (ie Pool id) ?

2 - Can we have multiple instance of the same query running on multiple KSQL SERVER ? if yes how does that work, eg with respect to consumer group or partitions ?

3 - Can we have multiple instance of the same query running on the same KSQL SERVER ?

Answer 1

@Robin Moffatt please correct if i am wrong:

1 - Let say you have the cli or confluent control center, does the cli connect to all server or one of them ? Can it send the query to multiple KSQL SERVER eg those with the same id (ie Pool id) ?

It seems like not. One connect to one KSQL SERVER, and the command will be forwarded to the Command Topic, which will be red by all KSQL SERVER.

2 - Can we have multiple instance of the same query running on multiple KSQL SERVER ? if yes how does that work, eg with respect to consumer group or partitions ?

Yes, the query is send to one KSQL SERVER, which forward it to the command topic, from which other KSQL SERVERs picks up and run. I suspect there must be coordination, to avoid over-provisioning the query. Otherwise if each Server Automatically run the command, we may end with more topology instance than there is partition. Can you clarify that last part ?

3 - Can we have multiple instance of the same query running on the same KSQL SERVER ?

No we can't. Only one instance of a query per server, however, that query my be set to have a certain number of Thread. Each Thread is " some sort of independent instance of the application ", or a unit of parallelism in Kafka Jargon. In fact i would say more specifically, each Thread is an independent instantiation of the Topology.