简体   繁体   中英

Write large byte arrays on RAID0 virtual disk

I have a camera that produces 20 frames per second. I made an app in C# that grabs the frames and put them in a concurrent queue. I have another thread that writes the frames on filesystem. Each frame is an array of 134184960 bytes that I must write on filesystem. The filesystem is an array of 20 SSD in RAID0 with theoretical speed of 500 MB/s * 20 with the bottleneck of PCiExpress 8x (7,7 GB/s) The production rate is about 2,5 GB/s, but my writing speed doesn't go over 900 MB/s I am using Filestream.Write ( link )

What can I do to increase my writing speed?

private ConcurrentQueue<CapturedFrame> cq = new ConcurrentQueue<CapturedFrame>();

....

try
{
    while (cq.TryDequeue(out BufferToWrite))
    {
        byte[] bufferInBytes = new byte[134184960];

        //Matrox function to extract bytes
        MIL.MbufGetColor(BufferToWrite.IDFrame, MIL.M_PLANAR, MIL.M_ALL_BANDS, bufferInBytes);
    
        using (FileStream fileStream = new FileStream(Path.Combine(AcquisitionFolder, DateTime.Now.ToString("yyyy-MM-dd-HHmmss-fffffff") + ".raw"), FileMode.Create))
        {
            fileStream.Write(bufferInBytes, 0, bufferInBytes.Length);
            fileStream.Close();
        }
    }   
}
catch (Exception ex)
{
    log.Error("Exception: " + ex.Message);
}

changed the FileStream to this with no improvement 65536 = 64K is the stripe size of the RAID0 array, I've set the lenght of the stream

using (FileStream fileStream = new FileStream(Path.Combine(AcquisitionFolder, DateTime.Now.ToString("yyyy-MM-dd-HHmmss-fffffff") + ".raw"), FileMode.Create, FileAccess.Write, FileShare.Read, 65536, FileOptions.SequentialScan))
                    {
                        fileStream.SetLength(3 * 8192 * 5460);
                        fileStream.Write(bufferInBytes, 0, bufferInBytes.Length);
                        fileStream.Close();
                    }

I was able to reach the 2,5 GB/s write speed with two steps:

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM