Download servlet throwing java heap space exception

Question

I have a search function in which i am returning all the records of the entities to a user on searh button , now matter how much the records are(right now, it is able to search 50,000 records). now i am trying to download all these records in a csv .if the records are less, then its working fine , but when its more than 30,000, it is throwing

Edited:-

Solution:- used these lines of code

            InputStream in = new ByteArrayInputStream(buffer.toString().getBytes("UTF-8"));
            ServletOutputStream out = response.getOutputStream();

           byte[] outputByte = new byte[4096];

             while(in.read(outputByte, 0, 4096) != -1)
                {
                   out.write(outputByte, 0, 4096);
                  }
         in.close();
            out.flush();
              out.close();*/

Answer 1

Instead of writing everything to a giant in memory buffer then making a giant in memory string copy of it, get the Writer from your HttpServletResponse and write the CSV directly to the client as you create it. This way you can flush the data down the network to the client and not have to keep two entire copies of it in RAM before sending the whole thing.

Alternately of course, make the heap bigger!

Answer 2

You should consider streaming the records over the socket rather than trying to buffer them all in memory. That would probably require you to pass in the OutputStream to your code that generates the csv.

Either that or run your server with more memory, but that's really not a good answer, because you're just putting off the OOM exception for another day.