Inputstream and Outputstream with reading a list of files

Question

I have this ArrayList files

for(File file : files){

    InputStream in = FileInputStream(file);
    // process each file and save it to file
    OutputStream out = FileOutputStream(file);
    try{

    } finally {
       in.close();
       out.close();
    }
}

the performance is really slow since every loop there is a in/out close(), is there a better way to do this? I tried to put outputstream oustide of the loop, it doesn't work.

Answer 1

A close() can take up to 20 ms. I doubt this is your program unless you have 1000's of files.

I suspect your performance problem is a lack of buffering the input and output. Can you show your buffering wrappers as well?

Answer 2

Using buffered streams makes a huge difference.

Try this:

for(final File file : files) {

    final InputStream in = new BufferedInputStream(new FileInputStream(file));
    final OutputStream out = new BufferedOutputStream(new FileOutputStream(new File(...)));
    try {
        // Process each file and save it to file
    }
    finally {
        try {
            in.close();
        }
        catch (IOException ignored) {}
        try {
            out.close();
        }
        catch (IOException ignored) {}
    }
}

Note that the IOException s that can be thrown when closing the streams must be ignored, or you will lose the potential initial exception.

Another problem is that both streams are on the same file, which doesn't work. So I suppose you're using two different files.

Answer 3

you can of course build a queue of OutputStreams and offload that to a background thread that handles the closing of these outputstreams. Same for InputStreams. Alternatively you can leave it down to the JVM to do that -- simply don't close the files and leave it to the GC to do that when objects are finalized.