Files downloaded as Binary with Java are corrupted

Question

I have written an downloader which should be used to download text files, as well as images. So I download the files as binaries. Many of the downloads work very well, but some parts of the text files and many image files are corrupted. The errors occur always at the same files and at the same places (as long as I can tell when analysing the text files). I used this code for downloading:

    public File downloadFile(HttpURLConnection connection) {
        return writeFileDataToFile(getFileData(connection));
    }     

    //downloads the data of the file and returns the content as string
    private List<Byte> getFileData(HttpURLConnection connection) {
        List<Byte> fileData = new ArrayList<>();

        try (InputStream input = connection.getInputStream()) {
            byte[] fileChunk = new byte[8*1024];
            int bytesRead;

            do {
                bytesRead = input.read(fileChunk);
                if (bytesRead != -1) {
                    fileData.addAll(Bytes.asList(fileChunk));
                    fileChunk = new byte[8*1024];
                }
            } while (bytesRead != -1);

            return fileData;
        } catch (IOException e) {
            System.out.println("Receiving file at " + url.toString() + " failed");
            System.exit(1);
            return null; //shouldn't be reached
        }
    }

    //writes data to the file
    private File writeFileDataToFile(List<Byte> fileData) {

        if (!this.file.exists()) {
            try {
                this.file.getParentFile().mkdirs();
                this.file.createNewFile();
            } catch (IOException e) {
                System.out.println("Error while creating file at " + file.getPath());
                System.exit(1);
            }
        }

        try (OutputStream output = new FileOutputStream(file)) {
            output.write(Bytes.toArray(fileData));
            return file;
        } catch (IOException e) {
            System.out.println("Error while accessing file at " + file.getPath());
            System.exit(1);
            return null;
        }
    }

Answer 1

I could suggest you to not pass through List of Byte, since you create a list of Byte from an array, to get it back to an array of Byte, which is not really efficient.

Moreover you wrongly assume the chunk size (not necesseraly 8192 bytes).

Why don't you just do something as:

private File writeFileDataToFile(HttpURLConnection connection) {
    if (!this.file.exists()) {
        try {
            this.file.getParentFile().mkdirs();
            //this.file.createNewFile(); // not needed, will be created at FileOutputStream
        } catch (IOException e) {
            System.out.println("Error while creating file at " + file.getPath());
            //System.exit(1);
            // instead do a throw of error or return null
            throw new YourException(message);
        }
    }
    OutputStream output = null;
    InputStream input = null;
    try {
      output = new FileOutputStream(file):
      input = connection.getInputStream();
      byte[] fileChunk = new byte[8*1024];
      int bytesRead;
      while ((bytesRead = input.read(fileChunk )) != -1) {
         output.write(fileChunk , 0, bytesRead);
      }
      return file;
    } catch (IOException e) {
      System.out.println("Receiving file at " + url.toString() + " failed");
      // System.exit(1); // you should avoid such exit
      // instead do a throw of error or return null
      throw new YourException(message);
    } finally {
      if (input != null) {
        try {
           input.close();
        } catch (Execption e2) {} // ignore
      }
      if (output != null) {
        try {
           output.close();
        } catch (Execption e2) {} // ignore
      }
    }
}

Answer 2

The failure was adding the whole fileChunk Array to file data, even if it wasn't completely filled by the read operation.

Fix:

//downloads the data of the file and returns the content as string
private List<Byte> getFileData(HttpURLConnection connection) {
    List<Byte> fileData = new ArrayList<>();

    try (InputStream input = connection.getInputStream()) {
        byte[] fileChunk = new byte[8*1024];
        int bytesRead;

        do {
            bytesRead = input.read(fileChunk);
            if (bytesRead != -1) {
                fileData.addAll(Bytes.asList(Arrays.copyOf(fileChunk, bytesRead)));
            }
        } while (bytesRead != -1);

        return fileData;
    } catch (IOException e) {
        System.out.println("Receiving file at " + url.toString() + " failed");
        System.exit(1);
        return null; //shouldn't be reached
    }
}

Where the relevant change is changing

if (bytesRead != -1) {
    fileData.addAll(Bytes.asList(fileChunk));
    fileChunk = new byte[8*1024];
}

into

 if (bytesRead != -1) {
    fileData.addAll(Bytes.asList(Arrays.copyOf(fileChunk, bytesRead)));
 }