如何以流形式获取httpclient响应

Question

我正在使用httpclient 4.5.5，我想获取高达1 GB的大文件作为响应。 但是好像

CloseableHttpResponse response = httpClient.execute(httpGet); HttpEntity entity = response.getEntity();

这将返回完整响应，因此在内存中具有完整响应不是很好。 有什么办法可以让响应作为流？

Answer 1

自版本4.0起的Apache HttpClient（以及Apache HttpAsyncClient）支持传入和传出HTTP消息的完整内容流。 使用HttpEntity来访问基础内容输入流

CloseableHttpClient client = HttpClients.createDefault();
HttpGet httpGet = new HttpGet("http://myhost/tons-of-stuff");
try (CloseableHttpResponse response1 = client.execute(httpGet)) {
    final HttpEntity entity = response1.getEntity();
    if (entity != null) {
        try (InputStream inputStream = entity.getContent()) {
            // do something useful with the stream
        }
    }
}

Answer 2

使用HttpURLConnection代替httpClient 。

final HttpURLConnection conn = (HttpURLConnection)url.openConnection();
final int bufferSize = 1024 * 1024;
conn.setChunkedStreamingMode(bufferSize);
final OutputStream out = conn.getOutputStream();

Answer 3

您需要Apache Async Client。

HttpAsyncClient是Apache HttpClient的ASYNC版本。 Apache HttpClient在内存中构造整个响应，而使用HttpAsyncClient时，您可以定义一个处理程序（消费者）以在接收数据时处理响应。

https://hc.apache.org/httpcomponents-asyncclient-4.1.x/index.html

这是他们官方示例代码中的一个示例

package org.apache.http.examples.nio.client;

import java.io.IOException;
import java.nio.CharBuffer;
import java.util.concurrent.Future;

import org.apache.http.HttpResponse;
import org.apache.http.impl.nio.client.CloseableHttpAsyncClient;
import org.apache.http.impl.nio.client.HttpAsyncClients;
import org.apache.http.nio.IOControl;
import org.apache.http.nio.client.methods.AsyncCharConsumer;
import org.apache.http.nio.client.methods.HttpAsyncMethods;
import org.apache.http.protocol.HttpContext;

/**
 * This example demonstrates an asynchronous HTTP request / response exchange with
 * a full content streaming.
 */
public class AsyncClientHttpExchangeStreaming {

    public static void main(final String[] args) throws Exception {
        CloseableHttpAsyncClient httpclient = HttpAsyncClients.createDefault();
        try {
            httpclient.start();
            Future<Boolean> future = httpclient.execute(
                    HttpAsyncMethods.createGet("http://httpbin.org/"),
                    new MyResponseConsumer(), null);
            Boolean result = future.get();
            if (result != null && result.booleanValue()) {
                System.out.println("Request successfully executed");
            } else {
                System.out.println("Request failed");
            }
            System.out.println("Shutting down");
        } finally {
            httpclient.close();
        }
        System.out.println("Done");
    }

    static class MyResponseConsumer extends AsyncCharConsumer<Boolean> {

        @Override
        protected void onResponseReceived(final HttpResponse response) {
        }

        @Override
        protected void onCharReceived(final CharBuffer buf, final IOControl ioctrl) throws IOException {
            while (buf.hasRemaining()) {
                System.out.print(buf.get());
            }
        }

        @Override
        protected void releaseResources() {
        }

        @Override
        protected Boolean buildResult(final HttpContext context) {
            return Boolean.TRUE;
        }

    }

}