Redis mget not working as expected with redis-py-cluster client

Question

I am using "redis-py-cluster" to connect to our Redis cluster. We have a requirement to query 100 keys from redis cluster in one call. I use redis hashtags (with {}) to hash all the keys to one node(Ex: {feed}1,{feed}2,{feed}3 are some sample keys).

I don't notice much of a time difference between querying 100 keys sequentially and querying 100 keys using mget.Sample code here:

rconn = RedisCluster(startup_nodes=self.startup_nodes, decode_responses=True)
flist=['{feed}1','{feed}2','{feed}3','{feed}4'....'{feed}100']
time1 = time.time()
for i in range(1,len(flist)):
    feed_s = rconn.get(flist[i])
print("Feeds get one by one time:%f", time.time()-time1)
time2 = time.time()
feed_m = rconn.mget(flist)
print("Feeds mget time:%f", time.time()-time2)

Does "redis-py-cluster" support mget? Is this the correct way of querying using mget?

versions: redis-cluster version: 5.0.7, python: 3.6, redis-py-cluster:2.0.0

Please help, Thanks!

Answer 1

Commands that works with multislot is not possible to use the regular implementation from redis-py, so other methods or workarounds have to be done in order to support mget commands the same way as the normal redis-py method works.

The mget command is one of these commands. If you look at the current cluster implementation of this method here https://github.com/Grokzen/redis-py-cluster/blob/master/rediscluster/client.py#L891 you will see that it works by mimicking the input and output from the upstream version from redis-py, but it changes the internal implementation as for each key you might have to talk with a different server.

So to enable the code to be a drop-in replacement with no changes to your code, the cluster mget will iterate over each key in sequence and get the value and return the data in the same format for a single server case. With this current implementation there will not be any performance gains but there is no other way to implement the method in any other way either that would give some performance gains, it was more important to implement the method to work the same, not to have the same performance.