Java 堆出 memory 问题，同时压缩 nio 路径列表

Question

I am asynchronously running the below method to zip the given set of nio paths. When there are multiple tasks running, java heap out of memory exception is encountered.

public InputStream compressToZip(String s3folderName, Set<Path> paths) throws Exception {
    try {
        ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
        ZipOutputStream zos = new ZipOutputStream(byteArrayOutputStream);
        paths.forEach(path -> {
            try {
                System.out.println("Zipping " + path.getFileName());
                zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
                FileInputStream ObjectInputStream = new FileInputStream(path.toFile());
                IOUtils.copy(ObjectInputStream, zos);
                zos.closeEntry();
            } catch (Exception e) {
                ...
            }
        });
        zos.close();
        return new ByteArrayInputStream(byteArrayOutputStream.toByteArray());
    } catch (Exception e) {
        ...
    }
}

The input stream returned from this file will be written on sftp location.

org.springframework.integration.file.remote.session.Session session = this.createSession(deliveryType, deliveryLocation);    
zippedIpStream = fileCompressionSvc.compressToZip(s3folderName, fileDir);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/"
                                + getfileNameFormat(fileNameFormat, masterId) + ".zip");

I am not sure what went wrong to occur java heap issue. Could you please assist me.

Answer 1

Changed the implementation to write the file into a file in local path and then send that file to sftp and then delete the temp zip file.

public void compressToZip(String s3folderName, Set<Path> distinctPaths, String efsPathWithFileName) throws Exception {
    try(FileOutputStream fos = new FileOutputStream(efsPathWithFileName);
        ZipOutputStream zos = new ZipOutputStream(fos)) {
        distinctPaths.forEach(path -> {
            try {
                zos.putNextEntry(new ZipEntry(path.getFileName().toString()));
                final FileInputStream fis = new FileInputStream(path.toFile());
                IOUtils.copy(fis, zos);
                zos.closeEntry();
            } catch (IOException e) {
                ...
            }
        });
    } catch (Exception e) {
        ...
        throw e;
    }
}

calling method:

InputStream zippedIpStream = new FileInputStream(tempCompressedFilePath);
session.write(zippedIpStream, deliveryLocation.getLocation().getFolder() + "/" + fileNameToWrite);
...
...                     
zippedIpStream.close();
...
...
Files.deleteIfExists(Paths.get(tempCompressedFilePath));