ZipFile constructor slow for zip64 file in Java/Apache Commons Compress

Question

I have to read file from zip file that contains over 100000 files and I am using Java 1.6 so I use Apache commons compress library for the time being. However, the constructor of ZipFile() takes around 30 seconds to return. (I admit the machine I am on is pretty out of date, an C2D E6550 with 8GB RAM, but if the zip file has less than 65535 files, the constructor returns almost immediately)

Now I need to speeding it up because our program opens zip files very frequently and the time it takes for ZipFile constructor is simply too long. I have two options: 1) split the zip file into volumes each with <65535 files or 2) cache ZipFile object and reuse it

However, both takes significant amount of refactoring/rewrite so before I go on with either solution, is there a way to speed up reading the zipfile with a different library or I am doing something wrong with Apache commons compress? This is how I create the zipfile object:

final File f =  new File(zipFileName);
if(f.exists() == false)
    throw new FileNotFoundException(zipFileName);
ZipFile zip = new ZipFile(f);

Any thoughts on how to attack this issue?

Answer 1

From looking at the source code , it is opening the zipfile and reading all the entries immediately. It stores this information in internal hashmaps which are not originally sized appropriately for your use case. All the rehashing could be the issue. But it is hard to be sure. You should execute it with a profiler to see where the time is being spent.

You could also grab the source for apache commons and make your own version which allows you to control the initial sizing to see if that helps.

If not, you could always construct this object in a separate thread and do something else useful while it is being constructed.