Low-latency communication of micro-services in remote, IPC and threading scenarios

Question

I want to create an ultra fast message processing C++ solution which will be CPU bound and micro-services based. It will process lots of request/response messages that are small enough (32 bytes to 1kb per message).

Some services will be placed in different hosts. Some will be on the same host, but in different processes. And some inside the same process, but in different threads.

Currently I'm thinking about communication protocols for such services topology. My first idea was to use TCP-based communication which allows to use loopback for IPC communication on the same host and even for threading communication. The benefit is to have a single communication code which allows to experiment with services topology. It will be easy to host service inside some process or move it to the remote host.

However I understand that if I want to have a low-latency solution with max RPS and speed of message delivery I have to split transport depending on communication type:

• Threads communication - the best results could be achieved with circular buffer or LMAX Disruptor pattern .

• IPC communication - I think pipes or circular buffer in shared memory are good solutions. Is there a better way to do IPC?

• Remote communication - async TCP server/client for sequential message delivery and UDP for multicasting.

Also I'm thinking about Linux solution, but having cross-platform would be great!

I believe Asio C++ Library is a good starting point for remote communications.

My questions are the following:

1. Is it worth creating a custom IPC/threading communication solutions or should I start with TCP-based localhost communications?
2. Can anyone provide me with some performance comparison results of different IPC techniques (locahost tcp vs pipes vs shared memory) for best RPS of small messages up to 1kb? (Eg shared memory will be 10-times faster than localhost TCP and 3-times faster than pipes or approximate RPS values to compare).
3. Maybe I missed some good low-lattency IPC/RPC technique or library that I should look into?
4. Or perhaps some production-ready solution exists for my problem that I can use or purchase the license?

Thanks in advance for your answers and suggestions!

---

IPC benchmarks

I just found and performed low-level IPC benchmarks under Linux (Linux ubuntu 4.4.0 x86_64 i7-6700K 4.00GHz). Message size was 128 bytes and messages count is 1000000. I get following results:

Pipe benchmark:

Message size:       128
Message count:      1000000
Total duration:     27367.454 ms
Average duration:   27.319 us
Minimum duration:   5.888 us
Maximum duration:   15763.712 us
Standard deviation: 26.664 us
Message rate:       36539 msg/s

FIFOs (named pipes) benchmark:

Message size:       128
Message count:      1000000
Total duration:     38100.093 ms
Average duration:   38.025 us
Minimum duration:   6.656 us
Maximum duration:   27415.040 us
Standard deviation: 91.614 us
Message rate:       26246 msg/s

Message Queues benchmark:

Message size:       128
Message count:      1000000
Total duration:     14723.159 ms
Average duration:   14.675 us
Minimum duration:   3.840 us
Maximum duration:   17437.184 us
Standard deviation: 53.615 us
Message rate:       67920 msg/s

Shared Memory benchmark:

Message size:       128
Message count:      1000000
Total duration:     261.650 ms
Average duration:   0.238 us
Minimum duration:   0.000 us
Maximum duration:   10092.032 us
Standard deviation: 22.095 us
Message rate:       3821893 msg/s

TCP sockets benchmark:

Message size:       128
Message count:      1000000
Total duration:     44477.257 ms
Average duration:   44.391 us
Minimum duration:   11.520 us
Maximum duration:   15863.296 us
Standard deviation: 44.905 us
Message rate:       22483 msg/s

Unix domain sockets benchmark:

Message size:       128
Message count:      1000000
Total duration:     24579.846 ms
Average duration:   24.531 us
Minimum duration:   2.560 us
Maximum duration:   15932.928 us
Standard deviation: 37.854 us
Message rate:       40683 msg/s

ZeroMQ benchmark:

Message size:       128
Message count:      1000000
Total duration:     64872.327 ms
Average duration:   64.808 us
Minimum duration:   23.552 us
Maximum duration:   16443.392 us
Standard deviation: 133.483 us
Message rate:       15414 msg/s

Answer 1

Maybe I missed some good low-lattency IPC/RPC technique or library that I should look into?

Continental published an IPC library that is focused on low latency and high bandwidth data transport on single hosts (using shared memory) or between different hosts (using udp multicast). It's running on windows and linux os. Take a look here https://github.com/continental/ecal

Answer 2

You wrote:

an ultra fast message processing C++ solution

That usually means getting hands into everything. Sounds like an interesting library in the end though if you pull it off.

Overall your question is (way) too broad - nevertheless here are my thoughts:

1. Hard to give any advice here...

2. Comparisons will be platform/system specific. Eg. TCP may be faster or slower depending on the system.

3. OpenMP and boost interprocess comes to mind.

4. You may wan't to look into or start out with for example apache thrift library (albeit its also cross-language - original developed for facebook backend servers i believe) you may do some early experimenting perhaps and get a feel for some issues to consider.