使用 RedisGears 对接收 pubsub 消息进行计算

Question

In my scenario I'm publishing messages to Redis, these messages contain GPS coordinates (latitude/longitude pairs).

Example:

redis-cli -p 16379 PUBLISH gps_positions "{'lat': 50.5243584, 'lon': 12.3616320}"
redis-cli -p 16379 PUBLISH gps_positions "{'lat': 50.5063360, 'lon': 12.3377472}"

A client subscribes to these messages:

redis-cli -p 16379 PSUBSCRIBE gps_positions

Now I'd like to calculate the bearing angle between the current and the previous GPS coordinate in the moment they are published to the Redis pubsub channel. The calculation should happen directly within Redis, transparent to the client. Since there is a huge amount of published messages I'd like to avoid saving any data within Redis - I only need the previous GPS coordinate for each new published coordinate.

In the end the client should receive the GPS positions including the bearing angle without having to do any calculations on its own:

"{'lat': 50.5243584, 'lon': 12.3616320, 'bearing': $BEARING}"

The algorithm to calculate the bearing does not matter for this question, but it looks like this:

import pyproj

def get_bearing(lat1, lon1, lat2, lon2):
    geodesic = pyproj.Geod(ellps='WGS84')
    fwd_azimuth, back_azimuth, distance = geodesic.inv(lat1, lon1, lat2, lon2)
    return fwd_azimuth, back_azimuth, distance

lat1, lon1 = 50.5243584, 12.3616320
lat2, lon2 = 50.5063360, 12.3377472

bearing = get_bearing(lat1, lon1, lat2, lon2)[0]

Is RedisGears able to listen and alter published messages before the clients receives these? Or can I use RedisGears to publish a message to a different channel containing the calculated bearing angle?

Answer 1

I'm not sure you can listen for PubSub events. However you could write to a stream that is only consumed by RedisGears and then either PUBLISH the modified event or maybe use Streams so you also get acknowledgement of these messages from consumers.

I wrote a demo that has this approach: https://github.com/hnsk/redis-streams-log-demo/blob/master/gears_functions.py

My test generator writes to a stream called "test", which is consumed by Gears with trimStream=True (default) so the messages get trimmed from the stream on consumption.

Then it takes the field "log_level" from the event and uses that as the key for a new stream and writes (XADD) the same data in it and that is then consumed by the actual consumers.

In the example it also writes the messages to a hash for RediSearch and a sorted set to count the events for each log_level.