Does the Polybase shard range constitute data knowledge?

Question

In the Polybase database , does the client encrypt and then determine in which shard a record belongs?

I was trying to understand how do you know the correct location of the data?

Answer 1

Yes, that's how it will work initially. You can use zk-snarks to prove that this is the correct location for the data. This is detailed further in the Polybase whitepaper .

There is also ongoing research into homomorphic encryption (computing over encrypted data without access to the secret key), which would allow this to happen on the indexer.