如何將大文件下載到字節數組中

Question

我正在嘗試使用 spring Rest模板下載大文件：

    private Result download(Long id) throws Exception {
    Result result = new Result();
    try {
        RestTemplate restTemplate = restTemplateFactory.getRestTemplate();
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        headers.set(AUTHORIZATION, authorizationBL.getAccessToken().getAccessToken());
        HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<MultiValueMap<String, String>>(null, headers);
        Map<String, Long> valuesMap = new HashMap<String, Long>(1);
        valuesMap.put(ID, id);
        ResponseEntity<byte[]> response = restTemplate.exchange(downloadURL, HttpMethod.GET, request, byte[].class);
        byte[] stream = response.getBody();
        result.setData(stream);
        result.setStatus(Result.Status.SUCCESS);
    } catch (Exception e) {
        throw e;
    }
    return result;
}

當文件大小很大時，可以說超過 100 MB，那么我遇到了問題：

java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236) ~[na:1.8.0_60]
at java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:191) ~[na:1.8.0_60]
at org.springframework.http.converter.ByteArrayHttpMessageConverter.readInternal(ByteArrayHttpMessageConverter.java:59) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.http.converter.ByteArrayHttpMessageConverter.readInternal(ByteArrayHttpMessageConverter.java:38) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.http.converter.AbstractHttpMessageConverter.read(AbstractHttpMessageConverter.java:193) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:105) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:924) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:908) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:662) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:620) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:538) ~[spring-web-4.3.13.RELEASE.jar:4.3.13.RELEASE]

我將堆大小增加到 10g，但也無濟於事。

Answer 1

不是將文件下載到字節數組中，而是將其保存到磁盤。 然后，您可以從程序中需要的任何其他地方讀取它。 例如，將您的方法更改為

/**
 * returns a File corresponding to the tmp file download for this resource
 */
private File download(Long id) throws IOException {
    File tmp = null;
    try {
        RestTemplate restTemplate = restTemplateFactory.getRestTemplate();
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_OCTET_STREAM);
        headers.set(AUTHORIZATION, authorizationBL.getAccessToken().getAccessToken());
        HttpEntity<MultiValueMap<String, String>> request = new HttpEntity<MultiValueMap<String, String>>(null, headers);
        ResponseEntity<byte[]> response = restTemplate.exchange(downloadURL, HttpMethod.GET, request, byte[].class);

        if (response.getStatusCode() == HttpStatus.OK) {
            tmp = Files.createTempFile("yourprefix", "yoursuffix");
            Files.write(tmp.toPath(), response.getBody());
        } else {
            // TODO handle this case
        }
    } catch (Exception e) {
        throw new IOException("An exception occurred while downloading", e);
    }
    return tmp;
}

一些觀察：

1. valuesMap未在任何地方使用（也已刪除）
2. result也被刪除

Answer 2

我能夠使用流而不是將文件存儲在 byte[] 中並使用IOUtils.copyLarge(inputStream, outputStream)來解決這個問題。