How to get all effective HTTP request headers?

Question

I want to use the new java.net.HttpClient to do some requests to another system.

For debug purposes I want to log (and later store in our db) the request that I send and the response that I receive.

How can I retrieve the effective http headers, that java is sending?

I tried to get the headers like this:

HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create("http://localhost:54113"))
        .build();

System.out.println("HTTP-Headers:\n---");
request.headers().map()
        .forEach((key, values) ->
                values.forEach(value ->
                        System.out.println(key + ": " + value)
                )
        );
System.out.println("---");

HttpClient.newHttpClient().send(request, HttpResponse.BodyHandlers.ofString());

But it outputs:

HTTP-Headers:
---
---

My server, however, tells me, that it receives these Http headers:

HTTP-Headers:
---
Connection: Upgrade, HTTP2-Settings
User-Agent: Java-http-client/11
Host: localhost:54113
HTTP2-Settings: AAEAAEAAAAIAAAABAAMAAABkAAQBAAAAAAUAAEAA
Content-Length: 0
Upgrade: h2c
---

I have a multithreaded application and simultanious requests might occur. Using a log framework with custom appenders is therefore probably not reliable.

Answer 1

I have an unfortunate answer to your question: Regrettably, impossible .

Some background on why this is the case:

The actual implementation of HttpRequest used by your average OpenJDK-based java-core-library implementation is not java.net.http.HttpRequest - that is merely an interface. It's jdk.internal.net.http.HttpRequestImpl .

This code has 2 separate lists of headers to send; one is the 'user headers' and the other is the 'system headers'. Your .headers() call retrieves solely the user headers, which are headers you explicitly asked to send, and, naturally, as you asked for none to send, it is empty.

The system headers is where those 6 headers are coming from. I don't think there is a way to get at these in a supported fashion. If you want to dip into unsupported strategies (Where you write code that queries internal state and is thus has no guarantee to work on other JVM implementations, or a future version of a basic JVM implementation), it's still quite difficult, unfortunately. Some basic reflection isn't going to get the job done here: It's the worst news imaginable:

• These 6 headers just aren't set, at all, until send is invoked. For example, the three headers that are HTTP2 related are set in the package-private setH2Upgrade method, and this method is passed the HttpClient object, which proves that this cannot possibly be called except in the chain of events started when you invoke send . An HttpClient object doesn't exist in the chain of code that makes HttpRequest objects, which proves this.

• To make matters considerably worse, the default HttpClient impl will first clone your HttpRequest, then does a bunch of ops on this clone (including adding those system headers), and then sends the clone, which means the HttpRequest object you have doesn't have any of these headers. Not even after the send call completes. So even if you are okay with fetching these headers after the send and are okay with using reflecting to dig into internal state to get em, it won't work.

You also can't reflect into the client because the relevant state (the clone of your httprequest object) isn't in a field, it's in a local variable, and reflection can't get you those.

A HttpRequest can be configured with custom proxies, which isn't much of a solution either: That's TCP/IP level proxies, not HTTP proxies, and headers are sent encrypted with HTTPS. Thus, writing code that (ab)uses the proxy settings so that you can make a 'proxy' that just bounces the connection around your own code first before sending it out, in order to see the headers in transit, is decidedly non-trivial.

The only solution I can offer you is to ditch java.net.http.HttpClient entirely and use a non-java-lib-core library that does do what you want. perhaps OkHttp . (Before you sing hallelujah, I don't actually know if OkHttp can provide you with all the headers it intends to send, or give you a way to register a hook that is duly notified, so investigate that first!)