Multi Thread zip4j

Question

zip4j is a great library. But i run into a problem when using it in a class that uses a thread. The zip4j method is called from a class that implements thread and sometimes (not always) it leaves files uncompress and somtimes there are leftofer files with the extension *.zip345. Also the process returns net.lingala.zip4j.exception.ZipException: cannot rename modified zip file.

The method zip4jProcess is called from the class public method. Class name is: SZipInterface.class

The SZipInterface.class is initialized in the thread class ex: ThreadObj.class and instantiated per thread. No static method is used.

What is the cause of the problems? How do you fix it? Is zip4j thread safe?

Method:

    private int zip4jProcess() {
    int status = 0;
    if (null != getInFiles() && getInFiles().length > 0) {
        for (String file : getInFiles()) {
            File sourceFile = new File(file);
            ZipFile zipFile = null;
            ZipParameters zipParams = new ZipParameters();
            if (getPassword() != null
                    && !getPassword().trim().equalsIgnoreCase("")) {
                zipParams.setPassword(getPassword());
                zipParams.setEncryptFiles(true);
                zipParams
                        .setEncryptionMethod(Zip4jConstants.ENC_METHOD_STANDARD);

            }
            zipParams
                    .setCompressionLevel(Zip4jConstants.DEFLATE_LEVEL_NORMAL);

            if (sourceFile.exists()) {
                try {
                    zipFile = new ZipFile(getZipFileName());
                    if (zipFile.getFile().exists()) {
                        zipFile.addFile(sourceFile, zipParams);
                        if (log.isDebugEnabled()) {
                            log.debug("Adding: " + sourceFile.getName()
                                    + " to " + zipFile.getFile().getName()
                                    + " Pass: " + getPassword());
                        }
                    } else {
                        zipFile.createZipFile(sourceFile, zipParams);
                        if (log.isDebugEnabled()) {
                            log.debug("Creating: " + sourceFile.getName()
                                    + " to " + zipFile.getFile().getName()
                                    + " Pass: " + getPassword());
                        }
                    }
                } catch (ZipException e) {
                    log.error(e);
                    status = 1;
                }
            }
        }
    }

    return status;
}

Answer 1

I believe the times where you have leftovers or uncomprossed files may be when multiple threads try to use the same zip file (probably at zipFile.addFile(...)).

So try handling the addFile differently with concurrency in mind.

Their support forum said it's tricky and not currently supported - see the link for the limitations of doing it.

This can be quite tricky to implement, if not impossible to achieve, especially when using encryption or when compressing the file (and not just using the store method, which just copies the source file to the zip without any compression). A current block of file being compressed/decompressed depends on the previous block. So, if multiple threads were to read or write, these threads cannot do this process simultaneously, but have to wait until the block n-1 (if n is the current block) is read/wrote. So, its as good as running the process in the same thread.

Writing different files in different threads to a zip file (each thread handling a unique file in the zip) can be tricky as well. For example: AES encryption requires a unique number (as part of salt calculation) for each file in the zip. And another example: if a zip file is being created and multiple number of files being added (with compression), then the second thread, which will start writing the second file to the zip should know exactly at which location in the zip file to start writing, and this cannot be determined until the first thread is done writing.

Some compression algorithms, like LZMA/LZMA2, support multithreading. Unfortunately, these compression methods are not supported by Zip4j at the moment.

Full text of their response (in case the post gets removed).