Base64 编码到解码文件转换问题

Question

I am processing very large files (> 2Gig). Each input file is Base64 encoded, andI am outputting to new files after decoding. Depending on the buffer size (LARGE_BUF) and for a given input file, my input to output conversion either works fine, is missing one or more bytes, or throws an exception at the outputStream.write line (IllegalArgumentException: Last unit does not have enough bits). Here is the code snippet (could not cut and paste so my not be perfect):

.
.

final int LARGE_BUF = 1024;
byte[] inBuf = new byte[LARGE_BUF];

try(InputStream inputStream = new FileInputStream(inFile); OutputStream outStream new new FileOutputStream(outFile)) {

   for(int len; (len = inputStream.read(inBuf)) > 0); ) {
      String out = new String(inBuf, 0, len);
      outStream.write(Base64.getMimeDecoder().decode(out.getBytes()));
   }
}

For instance, for my sample input file, if LARGE_BUF is 1024, output file is 4 bytes too small, if 2*1024, I get the exception mentioned above, if 7*1024, it works correctly. Grateful for any ideas. Thank you.

Answer 1

First, you are converting bytes into a String, then immediately back into bytes. So, remove the use of String entirely.

Second, base64 encoding turns each sequence of three bytes into four bytes, so when decoding, you need four bytes to properly decode three bytes of original data. It is not safe to create a new decoder for each arbitrarily read sequence of bytes, which may or may not have a length which is an exact multiple of four.

Finally, Base64.Decoder has a wrap(InputStream) method which makes this considerably easier:

try (InputStream inputStream = Base64.getDecoder().wrap(
    new BufferedInputStream(
        Files.newInputStream(Paths.get(inFile))))) {

    Files.copy(inputStream, Paths.get(outFile));
}