Unexpected closing of connection, when keep-alive is active

Question

I have some trouble with the HTTP connections on our server after a long running transaction. The next request will lead to an exception java.net.SocketException: Unexpected end of file from server .

It works nicely if I turn off either the keep-alive on the Apache HTTP server or on the JVM from the client.

Setup:

• Apache HTTP server, default config (also tried with increasing the keepalivetimeout to 60secs), connected to tomcat via AJP
• Apache tomcat server, default conf
• Java server application (Spring stack) answering HTTP requests inside tomcat
• Java client (Spring, RestTemplate) -> shooting HTTP calls against this server
• Java client can run on the same server (-> http against localhost) or on another machine, issue stays the same
• iptables disabled on the server
• Just one client was used for the test -> no other client connected to the server at this point in time

What happens?

...other stuff...
13:39:32.018 Do Rest call
13:39:32.111 Rest call return headers: Keep-Alive=[timeout=60, max=97], Connection=[Keep-Alive]
13:39:32.609 Do Rest call
13:39:33.849 Rest call return headers: Keep-Alive=[timeout=60, max=96], Connection=[Keep-Alive]
13:39:33.869 Do Rest call
...LONG RUNNING TRANSACTION...
13:40:05.828 Rest call return headers: Keep-Alive=[timeout=60, max=95], Connection=[Keep-Alive]
13:40:05.833 Do Rest call -> _java.net.SocketException: Unexpected end of file from server_; no trace in any log file on the server for this request

I cannot figure out what causes the connection to be dropped. I cannot find the last failing rest call in any server log. Any idea what it is or what I can do to further isolate the issue?

Here you can see the transactions logged on the server by tshark

Answer 1

server side has a connection timeout set so our keep alive connections would timeout after the allocated time and hence the intermittent errors.

remove keep-alive and try