亚马逊 sdk 从 spring 启动 Z93F725A074233D1C889F4F48B 中的 s3 存储桶下载大型 zip 文件

Question

我所做的是尝试从 s3 存储桶下载大数据 zip 文件

        S3ObjectInputStream inputStreams = s3object.getObjectContent(); 
        File newFile = new File(zipFileTempLocation + File.separator + CommonConstant.FILE_NAME);
        FileOutputStream fileOutputStream = new FileOutputStream(newFile);
        GZIPInputStream gzipInputStream = new GZIPInputStream(inputStream);
        LOGGER.info("staring to write {}", newFile.toPath());
        byte[] buffer = new byte[5000];
        int le
        while ((len = gzipInputStream.read(buffer)) > 0) {
            fileOutputStream.write(buffer, 0, len);
        }
        gzipInputStream.close();
        String newFileURl = newFile.getAbsolutePath();
        Path path = Paths.get(url);
        return Files.readAllBytes(path);

}

当我尝试运行我的服务时，它说堆外 memory 错误。 你能帮我解决这个问题吗？

Answer 1

您可以查看此答案和 stream 它，因此它不会加载到 memory 中。

  @GetMapping(value = "/downloadfile/**", produces = { MediaType.APPLICATION_OCTET_STREAM_VALUE })
    public ResponseEntity<S3ObjectInputStream> downloadFile(HttpServletRequest request) {
       //reads the content from S3 bucket and returns a S3ObjectInputStream
       S3Object object = publishAmazonS3.getObject("12345bucket", "/logs/file1.log");
       S3ObjectInputStream finalObject = object.getObjectContent();

        final StreamingResponseBody body = outputStream -> {
            int numberOfBytesToWrite = 0;
            byte[] data = new byte[1024];
            while ((numberOfBytesToWrite = finalObject.read(data, 0, data.length)) != -1) {
                System.out.println("Writing some bytes..");
                outputStream.write(data, 0, numberOfBytesToWrite);
            }
            finalObject.close();
        };
        return new ResponseEntity<>(body, HttpStatus.OK);
    }