thinking about scaling, i wonder how queries are distributed among KSQL Servers ?
1 - Let say you have the cli or confluent control center, does the cli connect to all server or one of them ? Can it send the query to multiple KSQL SERVER eg those with the same id (ie Pool id) ?
2 - Can we have multiple instance of the same query running on multiple KSQL SERVER ? if yes how does that work, eg with respect to consumer group or partitions ?
3 - Can we have multiple instance of the same query running on the same KSQL SERVER ?
@Robin Moffatt please correct if i am wrong:
1 - Let say you have the cli or confluent control center, does the cli connect to all server or one of them ? Can it send the query to multiple KSQL SERVER eg those with the same id (ie Pool id) ?
It seems like not. One connect to one KSQL SERVER, and the command will be forwarded to the Command Topic, which will be red by all KSQL SERVER.
2 - Can we have multiple instance of the same query running on multiple KSQL SERVER ? if yes how does that work, eg with respect to consumer group or partitions ?
Yes, the query is send to one KSQL SERVER, which forward it to the command topic, from which other KSQL SERVERs picks up and run. I suspect there must be coordination, to avoid over-provisioning the query. Otherwise if each Server Automatically run the command, we may end with more topology instance than there is partition. Can you clarify that last part ?
3 - Can we have multiple instance of the same query running on the same KSQL SERVER ?
No we can't. Only one instance of a query per server, however, that query my be set to have a certain number of Thread. Each Thread is " some sort of independent instance of the application ", or a unit of parallelism in Kafka Jargon. In fact i would say more specifically, each Thread is an independent instantiation of the Topology.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.