使用Java读写大型二进制文件

Question

So I have this code which reads the proper installer file from disk (tar.gz, exe or dmg), and streams it to the user (code below). The installers are actually archives, which can be extracted and setup can be ran manually (this is specific for Windows, Mac installer needs to be mounted, Unix installer also needs to be extracted)

    InputStream in = null;
    OutputStream out = null;

    byte[] buffer = new byte[16384];

    try {
        String bundle = ServletRequestUtils.getRequiredStringParameter(request, "bundle");

        String installerPath = constructFilePath(bundle);
        File installer = new File(installerPath);
        if(!installer.exists()){
            logger.error("Cannot read installer file");
            response.sendRedirect("/somewhere");
        }else{
            in = new FileInputStream(installer);
            response.setContentType(getBundleContentType(bundle)); //application/octet-stream or application/x-gzip or application/x-apple-diskimage
            response.setHeader("Pragma", "private"); 
            response.setHeader("Cache-Control", "private, must-revalidate"); 
            response.addHeader("Content-Disposition", "attachment;filename="+getBundleFileName(bundle)); //Setting new file name
            out = new BufferedOutputStream(response.getOutputStream());

            while((in.read(buffer)) != -1)
                out.write(buffer);
        }
    } catch (Exception e) {
        logger.error("Exception downloading installer file, reason: " + e);
        response.sendRedirect("/somewhere");
    } finally {
        if(in != null){
            in.close();
        }
        if(out != null){
            out.flush();
            out.close();
        }
    }

    return null;

I'll take the Windows (.exe) installer as an example. Previously, when I had the code to redirect to http:///somepath/installer.exe for the download, the file would've been downloaded and I was able to extract it with 7zip, but now, when I try to extract it with 7zip, I'm getting :

Cannot open file as archive.

However, I am able to double-click the .exe and successfully do the install. I am also able to extract it using winRAR as well.

Same thing happened with the Unix installer. When I download it to a Unix machine and try to extract it (by right-click "Extract here") I'm getting this error:

gzip: stdin: decompression OK, trailing garbage ignored 
/bin/gtar: Child returned status 2 
/bin/gtar: Error is not recoverable: exiting now

However, I am able to open it with "ark" and properly extract its contents.

I should also point out that the bytes of the files do not match after a download (the downloaded one compared to the one on the filesystem, which should be the same).

Am I missing something?

Answer 1

You could try write exactly the same data you read:

while ((read = in.read(buffer)) != -1) {
    out.write(buffer, 0, read);
}

Answer 2

It is because you are writing the whole buffer.

Imagine the file was 16385 bytes.

The first in.read(buffer) would fill up the whole buffer and return 16384. You will then write the whole buffer. The second time, it will read one byte and again, you will write the whole buffer.

Someone has beaten me to it, but I was going to add that you could use IOUtils to do this...

http://commons.apache.org/proper/commons-io/javadocs/api-release/org/apache/commons/io/IOUtils.html

For example IOUtils.copy(in, out)