将大量float []写入命名管道（或文件）

Question

I am looking for a way to push large amounts of data between 2 processes. The data chunks are blobs of a managed float[] array.

Currently I am using named pipes to move the data between both processes. It works perfectly fine. But unfortunately I seem to need to copy the float[]array into a byte array before I can use the NamedPipe.Write.

The blobs are >128MB each, so I prefer to find a solution that works without an extra copying buffer. Is there any way of achieving the same result without having to do the copy?

I was hoping for something like:

unsafe{
  fixed(float* fp = myfloatarray){
     pipe.UnsafeWrite(fp, ...)
  }
}

I did poke around to see if I could get the pipe file handle and use WriteFile pinvoke. It looks like messing around with the file handle of a live managed namedpipe instance is asking for trouble.

Any tips are very welcome!

Answer 1

Perhaps it makes more sense to use memory mapped files? https://msdn.microsoft.com/en-us/library/dd997372(v=vs.110).aspx

Answer 2

I had a look in the source code of the pipestream classes and found that internally the 'Write' methods just wraps around the Win32 WriteFile. So my guess is that the code below should work perfectly fine:

unsafe
{
    fixed(float* fp = myfloatarray)
    {
           WriteFile(pipe.SafePipeHandle,(byte*)fp,buffer.Length*sizeof(float), out written);
    }
 }

[DllImport("kernel32.dll", SetLastError = true)]
    static unsafe extern int WriteFile(
        SafeHandle handle,
        byte* bytes,
        uint numBytesToWrite,
        out int numBytesWritten,
        System.Threading.NativeOverlapped* lpOverlapped = null);