Fastest Way to Save 3D int Array? (XNA \ C#)

Question

I need a fast way to save a small 3D array to a file as quickly as possible. The array is 32x32x4 in size. I also need a fast way of reading the file.

So far I have tried looping through all the elements:

for (int xx = 0; xx < 32; xx += 1)
{
    for (int yy = 0; yy < 32; yy += 1)
    {
        for (int zz = 0; zz < 4; zz += 1)
        {
            String += FormatInt(array[xx, yy, zz]);
        }
    }
}

Turning each integer into a string with 2 digits: (The FormatInt() method above)

public string FormatInt(int num)
    {
        string String = "";
        String = Convert.ToString(num);
        int length = String.Length;
        for (int i = 0; i < (2 - length); i += 1)
        {
            String = String.Insert(0, "0");
        }
        return String;
    }

Then saving that string to a .txt file. I then load the file, then turn each 2-digit substring into an integer:

int Pos = 0;
            for (int xx = 0; xx < chunkSize; xx += 1)
            {
                for (int yy = 0; yy < chunkSize; yy += 1)
                {
                    for (int zz = 0; zz < 4; zz += 1)
                    {
                        array[xx, yy, zz] = Convert.ToInt32(String.Substring(Pos * 2, 2));
                        Pos += 1;
                    }
                }
            }

I need a faster way of saving the file. (A faster loading would be nice too, but it's not too slow right now.)

Answer 1

I would use a BinaryReader and BinaryWriter . Your current method is very inefficient at storage and is going to have memory problems appending so many strings with += (Use StringBuilder instead)

Saving:

using (BinaryWriter b = new BinaryWriter(File.Open("file.ext", FileMode.Create)))
{
    for (int xx = 0; xx < 32; xx += 1)
    {
        for (int yy = 0; yy < 32; yy += 1)
        {
            for (int zz = 0; zz < 4; zz += 1)
            {
                b.Write(array[xx, yy, zz]);
            }
        }
    }
}

Loading

using (BinaryReader b = new BinaryReader(File.Open("file.ext", FileMode.Open)))
{
    for (int xx = 0; xx < 32; xx += 1)
    {
        for (int yy = 0; yy < 32; yy += 1)
        {
            for (int zz = 0; zz < 4; zz += 1)
            {
                array[xx, yy, zz] = b.ReadInt32();
            }
        }
    }
}

This is far more effecient than writing a string to a file, you can event write Int16 s and Bytes if you have smaller data types. You can combine this with Gzip for very small files.

Answer 2

Cyral's answer is sound. But since you asked for the fastest possible way, my own tests indicate that using P/Invoke is about three times faster:

[DllImport("kernel32.dll")]
private static extern bool WriteFile(IntPtr hFile, IntPtr lpBuffer, int nNumberOfBytesToWrite, out int lpNumberOfBytesWritten, IntPtr lpOverlapped);

private static unsafe void Write(Int32[,,] array)
{
    fixed (int* pArray = array)
    {
        using (var file = File.Open("filename", FileMode.Create, FileAccess.Write))
        {
            int written;
            WriteFile(file.SafeFileHandle.DangerousGetHandle(), (IntPtr)pArray, array.Length, out writter, IntPtr.Zero);
        }
    }
}

There's a corresponding ReadFile() function in kernel32.dll that can be used for reading, mutatis mutandis .

Answer 3

如果要使用字符串，请使用StringBuilder而不是String + =，因为它更有效