Low-latency IPC between C++ and Java

Question

What is the best way to implement C++/Java IPC for the following situation?

(Someone recently asked a similar question , but my requirements are more specific)

1. I have two programs -- one written in C++, the other in Java -- that need to communicate with each other. Both are running on the same machine.

2. The programs send messages to each other. Messages are typically short (less than a few hundred bytes), but could potentially be 100KB or more in size.

3. Messages do not need to be acknowledged (ie, not a request/response model like HTTP). For example, the C++ program sends a message to the Java program, and the Java program may reply by sending a message to the C++ program at a later time -- and vice versa.

4. An ideal solution would have a) very low latency, b) no security hassles (user does not have to authorize ports to be opened etc.) and c) will be platform-agnostic.

My first thought was using sockets -- each program would act as a server to the other. Sockets have more overhead than other forms of IPC, and I don't know how the server would inform the client of the port number if I let the system auto-assign port numbers. I've also considered named pipes , but they are not supported (at least not consistently) across different platforms. JNI looks like an option, but can it cross process boundaries?

Any suggestions?

Thanks!

FOLLOW-UP QUESTIONS

1. If I go with sockets, would I need to open two sockets to allow for asynchronous communication as described above?

Answer 1

I'd suggest you to use TCP sockets .

The actual overhead of TCP sockets, as of my experience, is very very low compared to the other tasks' workload of the applications, at least the ones I use to develop. I mean, sometimes even if sockets' latency is twice as the latency of other IPC mechanisms, in the overall workflow they have very little impact. And it saves you the hassle of making IPC between a Java application and a C++ one, that will eventually require you to use a specific Java library that uses JNI, with the overhead of JNI and the one of the library itself.

I've actually measured, in my Java applications, that Garbage Collector impact is far more important than the latency caused by " loopback " TCP sockets.

Moreover, TCP sockets are more scalable (and portable.) than traditional IPC? What if in the future you'll have to run the client and the server on different machines, In the 'TCP sockets' scenario, you'll have to do a 5-minute hack, in the 'traditional IPC' scenario. you'll have to rewrite the whole IPC stuff.

However, what is the general workflow of your application?

Even if the acknowledgement is not required, I'd suggest to use TCP (and not UDP) to avoid unsorted delivery (which leads to pain in the ass when it comes to rearrange the stuff you received - some of your messages are 100KB and this doesn't fit in a UDP packet).

In reply to your last question, for the server to inform the client about the port, you can just make the server launch the client with a specific 'port' command line parameter, or make the server save a small file under /tmp (or another temporary directory) with the port number written inside.

Answer 2

I've heard good things about ZeroMQ for specifically these sort of scenarios. It even boasts to be faster than TCP in some cases. In short, certainly won't hurt to try it out.

Answer 3

An alternative is using memory mapped files and keeping it portable by checking compiler settings if you are posix or not. POSIX OS's have mmap() and in windows you would use CreateFileMapping()

In the boost library is a portable implementation for C++ and in java you should be able to use FileChannel() .

This page does a good job of explaining how this can be used for IPC http://en.wikipedia.org/wiki/Memory-mapped_file

Answer 4

When you say very low latency, you need to qualify that. You can send messages over a Socket loop back with a RTT of 20 microsecond. If this is fast enough, I would do that.

If this is not fast enough I would just place the C++ inside the Java application and call it via JNI. This would give you about 30 nano-second RTT.

The problem with using memory mapped data is getting the interlocking right. You might find a solution which works on one system, but might not work on others.

Answer 5

Using JNI would allow access to all of the possiblities in the system, not just those supported directly in Java; it would be necessary, for example, if you used shared memory. JNI is, itself, rather expensive, however.

The question of latency is tricky, because none of the mechanisms I know given any guarantees. All in all, the fastest would probably be some form of shared memory, using signals to wake up the other process when data is present. This would require using JNI on the Java side, but done correctly, would probably still give the lowest latency—doing it correctly (ensuring that no messages get lost) can be far from trivial, however. Unix based platforms do support queueing signals, and handling them as events in a separate thread; I don't know about Windows.

Other than that, a named pipe is generally pretty effective; the latency can be as good as shared memory, but it takes more time to get the data through (since it must be copied through the system). And it should be possible to access it directly from Java, without using JNI. Under Unix, it's also possible to configure sockets to respond just as quickly (in fact, this is what a named pipe is under the hood); I don't know if the Java interface supports these configuration options, however, and again, I don't know whether they're available under Windows.

Answer 6

An alternative would be to use an embedded DB (since you're considering multiple IPCs, I assume both the applications are on the same machine).

I use to work on an application earlier where the c++ application fetches data from all sorts of channels and put it in the DB (an in-memory database; TimesTen). In order to display that data to the user, the Java application would query it from the DB.

For your use, I don't know if you'd be willing to consider Oracle's Timesten, but you could as well use Berkeley's embedded DB.