算术运算导致溢出错误-对于c＃中的byte []数组

Question

i have this line in c# :

    byte[] bytes = new byte[streamReader.BaseStream.Length];

That Length returns a file size bigger than 4 GB.

at that line i have the Error below :

Arithmetic operation resulted in an overflow. 
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: System.OverflowException: Arithmetic operation resulted in an overflow.

Source Error: 


Line 41:             System.IO.BinaryReader br = new System.IO.BinaryReader(streamReader.BaseStream);
Line 42: 
Line 43:             byte[] bytes = new byte[streamReader.BaseStream.Length];
Line 44: 
Line 45:             br.Read(bytes, 0, (int)streamReader.BaseStream.Length);

how can i fix this error ?

edit
i am using .net 4
that code was part of a handler for download files like below :

using System;
using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.IO;
using WindowsServer.Classes;

namespace WindowsServer
{
    /// <summary>
    /// Summary description for HandlerForMyFE
    /// </summary>
    public class Handler : IHttpHandler, System.Web.SessionState.IRequiresSessionState
    {

        private HttpContext _context;
        private HttpContext Context
        {
            get
            {
                return _context;
            }
            set
            {
                _context = value;
            }
        }

        public void ProcessRequest(HttpContext context)
        {
            Context = context;
            string filePath = context.Request.QueryString["Downloadpath"];
            filePath = context.Server.MapPath(filePath);

            if (filePath == null)
            {
                return;
            }

            System.IO.StreamReader streamReader = new System.IO.StreamReader(filePath);
            System.IO.BinaryReader br = new System.IO.BinaryReader(streamReader.BaseStream);

            byte[] bytes = new byte[streamReader.BaseStream.Length];

            br.Read(bytes, 0, (int)streamReader.BaseStream.Length);

            if (bytes == null)
            {
                return;
            }

            streamReader.Close();
            br.Close();
            string fileName = System.IO.Path.GetFileName(filePath);
            string MimeType = GetMimeType(fileName);
            string extension = System.IO.Path.GetExtension(filePath);
            char[] extension_ar = extension.ToCharArray();
            string extension_Without_dot = string.Empty;
            for (int i = 1; i < extension_ar.Length; i++)
            {
                extension_Without_dot += extension_ar[i];
            }

            string filesize = string.Empty;
            FileInfo f = new FileInfo(filePath);
            filesize = f.Length.ToString();

            //DownloadFile.DownloadFileMethod_2(Context, filePath, 5242880);
              WriteFile(bytes, fileName, filesize, MimeType + " " + extension_Without_dot, context.Response);
        }

       private void WriteFile(byte[] content, string fileName, string filesize, string contentType, HttpResponse response)
    {
        response.Buffer = true;
        response.Clear();

        response.ContentType = contentType;

        response.AddHeader("content-disposition", "attachment; filename=" + fileName);

        response.AddHeader("Content-Length", filesize);

        response.BinaryWrite(content);
        response.Flush();
        response.End();
    }

        private string GetMimeType(string fileName)
        {
            string mimeType = "application/unknown";
            string ext = System.IO.Path.GetExtension(fileName).ToLower();
            Microsoft.Win32.RegistryKey regKey = Microsoft.Win32.Registry.ClassesRoot.OpenSubKey(ext);
            if (regKey != null && regKey.GetValue("Content Type") != null)
                mimeType = regKey.GetValue("Content Type").ToString();
            return mimeType;
        }

        public bool IsReusable
        {
            get
            {
                return false;
            }
        }
    }
}

thanks in advance

Answer 1

No array in .NET can hold more than 2^31 element ( System.Int32.MaxValue ) or a max size of 2 GB which roughly would make for a 2 GB byte array.

For a workaround see http://blogs.msdn.com/b/joshwil/archive/2005/08/10/450202.aspx

Another option is to use MemoryMappedFile and Streams on those - this will alow to access a file of any size...

To make that download code work you could either read a chunk, send it, read the next chunk etc. OR use a Stream for reading and write to the OutputStream without any intermediate buffer...

Another option is to use TransmitFile which can handle files > 4 GB.

EDIT - as per comment:

You could just replace the code after

if (filePath == null)
{
return;
}

with

response.Clear();

response.ContentType = GetMimeType (System.IO.Path.GetFileName(filePath));

response.AddHeader("content-disposition", "attachment; filename=" + System.IO.Path.GetFileName(filePath));

response.TransmitFile (filePath);    

OR with

long FileL = (new FileInfo(filePath)).Length;
byte[] bytes = new byte[1024*1024];

response.Clear();

response.ContentType = GetMimeType (System.IO.Path.GetFileName(filePath));

response.AddHeader("content-disposition", "attachment; filename=" + System.IO.Path.GetFileName(filePath));

response.AddHeader("Content-Length", FileL.ToString());

using (FileStream FS = File.OpenRead(filePath))
{
int bytesRead = 0;
while ((bytesRead = FS.Read (bytes, 0, bytes.Length)) > 0 )
{
response.OutputStream.Write(bytes, 0, bytesRead);
response.Flush();
};

response.Close();
}

Answer 2

You're going to have to use a smaller buffer and read the stream piecemeal. I don't think you're going to be able to create a byte array that's bigger than 4 GB.

If you're using .NET 4, you can use the new Stream.CopyTo(Stream) instance method to copy the input stream to an output stream.

Answer 3

In .NET the largest object you could allocate is around 2GB. Quote from the documentation :

As with 32-bit Windows operating systems, there is a 2GB limit on the size of an object you can create while running a 64-bit managed application on a 64-bit Windows operating system.

There is also a blog post from the CLR team.

Of course 2GB is the theoretical limit. The practical limit is lower. So you will have to read this in chunks:

const int ChunkSize = 1024 * 1024 * 4; // process in chunks of 4MB:
using (var br = new BinaryReader(streamReader.BaseStream))
{
    var buffer = new byte[ChunkSize];
    int bytesRead;
    while ((bytesRead = br.Read(buffer, 0, buffer.Length)) > 0)
    {
        // TODO: process the chunk which is the buffer array from 0 to bytesRead
    }
}

Also your code seems a bit strange. You have a StreamReader which processes encoded strings and yet you are working with byte arrays and BinaryReaders. Seems kinda strange.