Most efficient way to transfer images continuously across the network

Question

I have an architecture in which there are separate services which run independently. My one of the service is continuously fetching frames from camera and sending them to another service which performs some processing on the frame (like face detection, face recognition etc) and sends back the results. Services can be run on different machines.

Please suggest any good library or something which is fast at transferring frames between services. I have already a few options in my mind like Kafka and ZeroMq but I am also confused between them, which to choose.

Any good pipeline design is also welcome. Thanks

Answer 1

After experimenting with the Kafka pipeline in a very similar scenario my suggestion is that Kafka is not a suitable choice for it. Since the frequency and size of messages are the concern here while dealing with the live image streams.

The choices that can be tested are:

• Zero MQ
• Rabbit MQ

Zero MQ will highly likely perform well in the given scenario.