Java: How to count read bytes from InputStream without allocating the full memory before

Question

I have a Java-backend where user can upload files to it. I want to limit these uploaded files to a max size and want to check the amount of uploaded bytes while the upload happens and break the transmission as soon as the limit is reached.

Currently I am using InputStream.available() before allocation for determination of estimated size, but that seems to be seen as unreliable.

Any suggestions?

Answer 1

You can use Guava's CountingInputstream or Apache IO's CountingInputStream when you want to know how many bytes have been read.

On the other hand when you want to stop the upload immediatly when reaching some limit then just count while reading chunks of bytes and close the stream when the limit has been exceeded.

Answer 2

int count = 1;
InputStream stream;
if (stream.available() < 3) {
count++;
}
Result:
[0][1]{2][3] 
 1  1  1  1

Answer 3

You don't have to 'allocat[e] the full memory before'. Just use a normally sized buffer, say 8k, and perform the normal copy loop, tallying the total transferred. If it exceeds the quota, stop, and destroy the output file.

Answer 4

If you're using a servlet and a multipart request you can do this:

public void doPost( final HttpServletRequest request, final HttpServletResponse response )
    throws ServletException, IOException {
  String contentLength = request.getHeader("Content-Length");
  if (contentLength != null && maxRequestSize > 0 && 
           Integer.parseInt(contentLength) > maxRequestSize) {
     throw new MyFileUploadException("Multipart request is larger than allowed size");
  }
}

Answer 5

My solution looks like this:

public static final byte[] readBytes (InputStream in, int maxBytes)
throws IOException {
    byte[] result               = new byte[maxBytes];
    int bytesRead               = in.read (result);
    if (bytesRead > maxBytes) {         
        throw new IOException   ("Reached max bytes (" + maxBytes + ")");
    }       
    if (bytesRead < 0) {            
        result                  = new byte[0];
    }
    else {
        byte[] tmp              = new byte[bytesRead];
        System.arraycopy        (result, 0, tmp, 0, bytesRead);
        result                  = tmp;
    }       
    return result;
}

EDIT: New variant

public static final byte[] readBytes (InputStream in, int bufferSize, int maxBytes)
throws IOException {

    ByteArrayOutputStream out       = new ByteArrayOutputStream();
    byte[] buffer                   = new byte[bufferSize];

    int bytesRead                   = in.read (buffer);
    out.write                       (buffer, 0, bytesRead);

    while (bytesRead >= 0) {

        if (maxBytes > 0 && out.size() > maxBytes) {

            String message          = "Reached max bytes (" + maxBytes + ")";
            log.trace               (message);
            throw new IOException   (message);
        }

        bytesRead                   = in.read (buffer);

        if (bytesRead < 0)
            break;

        out.write                   (buffer, 0, bytesRead);
    }

    return out.toByteArray();
}

Answer 6

All method implementations of read return the number of bytes read. So you can initiate a counter and increment it appropriately with each read to see how many bytes you've reads so far. Method available() allows you to see how many bytes are available for reading at the buffer at the moment and it has no relation to the total size of the file. this method could be very useful though to optimize your reading so each time you can request to read the chunk that is readily available and avoid blocking. Also in your case you can predict before reading if the amount of bytes that you will have after the upcoming reading will exceed your limit and thus you can cancel it even before you read the next chunk