download large files using servlet

Question

I am using Apache Tomcat Server 6 and Java 1.6 and am trying to write large mp3 files to the ServletOutputStream for a user to download. Files are ranging from a 50-750MB at the moment.

The smaller files aren't causing too much of a problem but with the larger files it and getting socket exception broken pipe.

File fileMp3 = new File(objDownloadSong.getStrSongFolder() + "/" + strSongIdName);
FileInputStream fis = new FileInputStream(fileMp3);
response.setContentType("audio/mpeg");
response.setHeader("Content-Disposition", "attachment; filename=\"" + strSongName + ".mp3\";");
response.setContentLength((int) fileMp3.length());
OutputStream os = response.getOutputStream();

try {
    int byteRead = 0;
    while ((byteRead = fis.read()) != -1) {
        os.write(byteRead);
    }
    os.flush();
} catch (Exception excp) {
    downloadComplete = "-1";
    excp.printStackTrace();
} finally {
    os.close();
    fis.close();
}

Answer 1

but with the larger files it appears that it is being written into the heap which is then causing an OutOfMemory error and bringing down the entire server

The cause lies somewhere else than in the as far given code snippet. One of the causes would be reading the entire file into a byte[] , but that doesn't seem to happen in the code you posted. Also, Tomcat 6 by default auto-flushes the response stream on every 2KB. In the future please include the entire stacktrace in the question as well. It might indicate a HttpServletResponseWrapper and/or a Filter in the chain which is possibly buffering the entire response.

also getting socket exception broken pipe.

This just means that the other side has aborted the request. Nothing to do against from the server side on and it should technically also not harm. You can safely ignore it.

Answer 2

You are writing out files that are 3/4s of a gig one byte at a time?

Try using a bigger buffer.

int BUFF_SIZE = 1024;
byte[] buffer = new byte[BUFF_SIZE];
File fileMp3 = new File(objDownloadSong.getStrSongFolder() + "/" + strSongIdName);
FileInputStream fis = new FileInputStream(fileMp3);
response.setContentType("audio/mpeg");
response.setHeader("Content-Disposition", "attachment; filename=\"" + strSongName + ".mp3\";");
response.setContentLength((int) fileMp3.length());
OutputStream os = response.getOutputStream();
int byteCount = 0;
try {
    do {
        byteCount = fis.read(buffer);
        if (byteCount == -1)
           break;
        os.write(buffer, 0, byteCount);
        os.flush();
    } while (true);
}
} catch (Exception excp) {
    downloadComplete = "-1";
    excp.printStackTrace();
} finally {
    os.close();
    fis.close();
}

BTW, before I started to answer this question I started an empty loop that will loop 750,000,000,000 (your large file size) times to see how long that would take. It's still running.

Answer 3

One common problem with downloading large file is that you are buffering the whole file, which can cause broken pipe if client is not patient enough. You may also run out of memory on server when the file is too big.

Try to force chunked encoding by doing this,

response.setContentLength(-1);

This will cause the connector to stream the file chunk by chunk.