Sending large volume of data between two C# programs

Question

I am currently working on a C# (I could use python also) plug-in for two separate programs that need to communicate. In the first first program, I deconstruct 3D geometry into edges, points, normals, etc. Then I send all of this data to my plug-in in my second program to be rebuilt. Ideally this would happen as fast as possible to keep things i "real time".

Currently, I am converting my data with JSON, and writing the JSON to the disk. Then my second program watches for file changes, and then reads the file and uses the JSON data.

By far the biggest bottle neck of my entire plugin is the read/write process. There has to be a faster way than writing to a file.

Answer 1

There are several ways to use interprocess communication.
The most well known are used between different machines: WCF(.NET 3.5) and Remoting(.NET 2)

For on-machine communication you can choose to use Named pipes or Memory mapped files.

Memory mapped files are similar to your solution in that they use the page file as a backup.

I think the Named pipes solution is the most convenient: You set up a "server" stream and wait for some "client" to connect. Then you transfer the data just as you would through any other stream.

Here's NamedPipeServerStream .
And this is NamedPipeClientStream .
The code example there pretty much covers it.

Answer 2

I think WCF with named pipes would do the job, you just have to create transfer objects, which will be serialized and it will be all done by WCF automagically, or you can just prepare your existing objects to be transfered by named pipe with not really big overhead. Using json would be nice but it creates additional layer, and with WCF you transfer objects which can be used right away without translation by json. (Really they are translated to xml but you are not doing it by yourself so it is better then you could do with parsing json I think).