Fastest way to send long string from C# to Python

Question

I have a long string of data streaming continuously, which is generated by my C# script. I would like to use it as an input to Python Machine Learning model.

What is the fastest & easiest way to implement this transmission?

I have found some possible options:

1. Use sockets.
2. Use http.
3. Write data to a file from C#, and read the file from Python

4. Data as an argument, while python is running as a process in C# or

5. Something different?

Answer 1

Provided that these processes run on the same machine, I think when handling big chunks of data between process memory mapping is the way to go.

In other case TCP/UDP sockets is a good option since they don't have the overhead of the Application layer which, HTTP for example, has

Answer 2

I suggest option 6 Use Database like sqlite.

With this option you gain:

• Both running OnLine
• Python set flags to these records in database when they processed.
• You can complete operation latter if python off, and continue processing data.
• You can add results from python to the database.
• you have a history in database you may need latter for post processing.