How to gunzip data on the fly as i'm reading it from an InputStream to an OutputStream?

Question

I have a large InputStream containing gzipped data.

I cannot modify the data in the InputStream directly. Code that uses this InputStream later on expects unmodified, compressed data. I could swap out the InputStream with a new InputStream if needed, but the data must remain compressed.

I need to print out the uncompressed contents of the InputStream as I go for debugging purposes.

What is the simplest way to print the uncompressed data in my InputStream to a PrintStream , without irrevocably uncompressing the InputStream itself and without reading the whole thing into memory?

Answer 1

Here's how I did it.

// http://stackoverflow.com/a/12107486/82156
public static InputStream wrapInputStreamAndCopyToOutputStream(InputStream in, final boolean gzipped, final OutputStream out) throws IOException {
    // Create a tee-splitter for the other reader.
    final PipedInputStream inCopy = new PipedInputStream();
    final TeeInputStream inWrapper = new TeeInputStream(in, new PipedOutputStream(inCopy));

    new Thread(Thread.currentThread().getName() + "-log-writer") {
        @Override
        public void run() {
            try {
                IOUtils.copy(gzipped ? new GZIPInputStream(inCopy) : inCopy, new BufferedOutputStream(out));
            } catch (IOException e) {
                Log.e(TAG, e);
            }
        }
    }.start();
    return inWrapper;
}

This method wraps the original InputStream and returns the wrapper, which you'll need to use from now on (don't use the original InputStream). It then uses an Apache Commons TeeInputStream to copy data to a PipedOutputStream using a thread, optionally decompressing it along the way.

To use, simply do something like the following:

InputStream inputStream = ...; // your original inputstream
inputStream = wrapInputStreamAndCopyToOutputStream(inputStream,true,System.out); // wrap your inputStream and copy the data to System.out

doSomethingWithInputStream(inputStream); // Consume the wrapped InputStream like you were already going to do

The background thread will stick around until the foreground thread consumes the entire input stream, buffering the output in chunks and periodically writing it to System.out until it's all done.