How to get a byte array from FileInputStream without OutOfMemory error

Question

I have a FileInputStream which has 200MB of data. I have to retrieve the bytes from the input stream.

I'm using the below code to convert InputStream into byte array.

private byte[] convertStreamToByteArray(InputStream inputStream) {
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    try {
        int i;
        while ((i = inputStream.read()) > 0) {
            bos.write(i);
        }
    } catch (IOException e) {
        e.printStackTrace();
    }
    return bos.toByteArray();
}

I'm getting OutOfMemory exception while coverting such a large data to a byte array.

Kindly let me know any possible solutions to convert InputStream to byte array.

Answer 1

Why do you want to hold the 200MB file in memory? What are you going to to with the byte array?

If you are going to write it to an OutputStream, get the OutputStream ready first, then read the InputStream a chunk at a time, writing the chunk to the OutputStream as you go. You'll never store more than the chunk in memory.

eg:

     public static void pipe(InputStream is, OutputStream os) throws IOException {

        int read = -1;
        byte[] buf = new byte[1024];

        try {
            while( (read = is.read(buf)) != -1) {
                os.write(buf, 0, read);
            }
        }
        finally {
            is.close();
            os.close();
        }
    }

This code will take two streams and pipe one to the other.

Answer 2

Android application has limited Heap Memory and which depend on devices. Currently most of the new devices has 64 but it could be more or less depend on Manufacturer. I have seen device come come with 128 MB heap Memory.

So what this really mean?

Its simply means that regardless of available physical memory your application is not allowed to grow more then allocated heap size.

From Android API level 11 you can request for additional memory by using manifest tag android:largeHeap="true" which will be double your heap size. That simply means if your devices has 64 you will get 128 and in case of 128 you will get 256. But this will not work for lower API version.

I am not exactly sure what is your requirement, but if you planning to send over HTTP then read file send data and read again. You can follow the same procedure for file IO also. Just to make sure not to use memory more then available heap size. Just to be extra cautious make sure you leave some room for application execution.

Answer 3

Your problem is not about how to convert InputStream to byte array but that the array is to big to fit in memory. You don't have much choice but to find a way to process bytes from InputStream in smaller blocks.

Answer 4

You'll probably need to massively increase the heap size. Try running your Java virtual machine with the -Xms384m -Xmx384m flag (which specifies a starting and maximum heap size of 384 megabytes, unless I'm wrong). See this for an old version of the available options: depending on the specific virtual machine and platform you may need to do some digging around, but -Xms and -Xmx should get you over that hump.

Now, you probably really SHOULDN'T read it into a byte array, but if that's your application, then...

Answer 5

try this code

private byte[] convertStreamToByteArray(InputStream inputStream) {
ByteArrayOutputStream byteOutStream = new ByteArrayOutputStream();
                    int readByte = 0;
        byte[] buffer = new byte[2024];

                    while(true)
                    {
                        readByte = inputStream.read(buffer);
                        if(readByte == -1)
                        {
                            break;
                        }
                        byteOutStream.write(buffer);
                    }
                    inputStream.close();
                    byteOutStream.flush();
                    byteOutStream.close();
                    byte[] byteArray= byteOutStream.toByteArray();
                    return byteArray;
}

try to read chunk of data from InputStream .