通過 Spring REST 模板下載大文件

Question

服務器代碼：

@POST
@Path("reportDownload")
@Consumes(MediaType.APPLICATION_JSON)
public Response generateReport(QueryData queryData) {
     File file = new File("report.xlsx") // large file
     StreamingOutput stream = new FileStreamingOutput(file) ; 
        return Response.ok(stream, MediaType.APPLICATION_OCTET_STREAM)
            .header("filename" , file.getName())
            .build();
}

客戶代碼：

使用以下代碼，我可以下載文件達到一定的限制。 出現大文件的內存堆錯誤。

final String uri = buildUri("/reportGenerate/reportDownload");

    HttpComponentsClientHttpRequestFactory factory = new HttpComponentsClientHttpRequestFactory();
    factory.setReadTimeout(read_timeout);
    factory.setConnectTimeout(connection_timeout);

    RestTemplate restTemplate = new RestTemplate(factory);

    HttpHeaders headers = new HttpHeaders();
    headers.setContentType(MediaType.APPLICATION_JSON);
    List<MediaType> mediaTypeList = new ArrayList<>();
    mediaTypeList.add(MediaType.APPLICATION_OCTET_STREAM);
    headers.setAccept(mediaTypeList);
    HttpEntity entity = new HttpEntity(queryData, headers);
    ResponseEntity<byte[]> data = restTemplate.exchange(uri, HttpMethod.POST, entity, byte[].class);
    HttpHeaders responseHeader = data.getHeaders();
    String fileName = (String) responseHeader.get("filename").get(0);
    String downloadFolder = ApplicationConfig.REPORT_DOWNLOAD_FOLDER.getValue();
    if (data.getStatusCode() == HttpStatus.OK) {
        FileOutputStream fos = null;
        File toFile = null;
        try {
            toFile = new File(downloadFolder + File.separator + fileName);
            fos = new FileOutputStream(toFile);
            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            IOUtils.write(data.getBody(), bos);
            bos.writeTo(fos);

        } catch (Exception e) {
            convertReportException(e);
        } finally {
            if (fos != null) {
                try {
                    fos.close();
                } catch (IOException ex) {
                    convertReportException(ex);
                }
            }
            return toFile;
        }
    }

如何使用流下載更大的文件。

Answer 1

這是我使用ResponseExtractor的方法。 基於這個Spring Jira issue的提示。

RestTemplate restTemplate // = ...;

// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
        .setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));

// Streams the response instead of loading it all in memory
ResponseExtractor<Void> responseExtractor = response -> {
    // Here I write the response to a file but do what you like
    Path path = Paths.get("some/path");
    Files.copy(response.getBody(), path);
    return null;
};
restTemplate.execute(URI.create("www.something.com"), HttpMethod.GET, requestCallback, responseExtractor);

更新

這是RestTemplate在postForObject和朋友的幕后所做的（我的內聯評論）：

@Override
public <T> T postForObject(String url, Object request, Class<T> responseType, Map<String, ?> uriVariables)
        throws RestClientException {

    // From RequestCallback's javadoc:
    // Callback interface for code that operates on a ClientHttpRequest.
    // Allows to manipulate the request headers, and write to the request body. 
    //
    // Used internally by the RestTemplate, but also useful for application code.
    RequestCallback requestCallback = httpEntityCallback(request, responseType);

    // HttpMessageConverterExtractor checks the response type header and requested
    // responseType class to select the proper message converter to handle the response.
    // It also implements ResponseExtractor.
    HttpMessageConverterExtractor<T> responseExtractor =
            new HttpMessageConverterExtractor<T>(responseType, getMessageConverters(), logger);
    return execute(url, HttpMethod.POST, requestCallback, responseExtractor, uriVariables);
}

/**
 * Returns a request callback implementation that writes the given object to the
 * request stream.
 */
protected <T> RequestCallback httpEntityCallback(Object requestBody, Type responseType) {
    return new HttpEntityRequestCallback(requestBody, responseType);
}

注意：這本質上是我在https://stackoverflow.com/a/38664475/1030527上的答案的副本，但我不能將問題標記為重復，因為這一個或那個都沒有投票贊成答案。

Answer 2

在您提到的client處
不要將文件存儲在memory中以通過RestTemplate下載大文件，否則會導致Java heap exception 。
它應該存儲在disk 。

這是一些通過RestTemplate下載大文件的代碼示例

@GetMapping("largeFile")
    public ResponseEntity<InputStreamResource> downloadLargeFile(
            @RequestParam("fileName") String fileName
    ) throws IOException {

        RestTemplate restTemplate = new RestTemplate();

        // Optional Accept header
        RequestCallback requestCallback = request -> request.getHeaders()
                .setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));

        // Streams the response instead of loading it all in memory
        ResponseExtractor<InputStreamResource> responseExtractor = response -> {
            // Here I write the response to a file but do what you like
            Path path = Paths.get("tmp/" + fileName);
            Files.copy(response.getBody(), path, StandardCopyOption.REPLACE_EXISTING);
            return new InputStreamResource(new FileInputStream(String.format("tmp/%s", fileName)));
        };

        InputStreamResource response = restTemplate.execute(
            String.format("http://%s:%s/file/largeFileRestTemplate?fileName=%s", host, "9091", fileName),
            HttpMethod.GET,
            requestCallback,
            responseExtractor
        );

        return ResponseEntity
            .ok()
            .header(HttpHeaders.CONTENT_DISPOSITION, String.format("attachment; filename=%s", fileName))
            .body(response);
    }