How to keep Java sockets low-latency in a loop using Http persistent connection?

Question

I'm using a program in java that is repeatedly downloading some data from an API. I need this data approx. every 1 sec and low-latency is important for my application! So I made something like this (pseudo-code):

while (true)
   1: download from API
   2: sleep from remainder of 1 sec
end

The average download time is 40-50ms. When I omit the sleeptime (step 2) I get an average download time of 6-8ms.

I've tried this in many different implementations: HttpUrlConnection, HttpClient (Java11) and Apache HttpClient, but the problems remains. So I made my own socket implementation but the problem persists. Is there a way to fix this?

I have tried TCP_NO_DELAY but without results.

My socket implementation:

try (Socket socket = new Socket(hostname, 80)) {
    OutputStream writer = socket.getOutputStream();                     
    InputStream input = socket.getInputStream();                       

    for (int i = 0; i < 10; i++) {                   
        //request headers
        String request = "GET " + path + " HTTP/1.1"    + "\r\n" 
                        + "Host: " + hostname           + "\r\n"
                        + "Connection: keep-alive"      + "\r\n"
                        + "\r\n";
        byte[] wb = request.getBytes(StandardCharsets.US_ASCII);        
        writer.write(wb);

        String statusLine       = readInputStatusLine(input);
        LinkedHashMap headers   = readInputHeaders(input);
        String body             = readBody(headers, input);

        //SLEEP FOR A WHILE (THIS CAUSES DELAYS!!??)
        Thread.sleep(1000);                                             
    }
    socket.close();                                                    
}

Answer 1

Another-dave was right! This issue is being caused by some server-side, maybe caching, issue. I was able to test this and make a work-around by doing 2 subsequent polls and than sleep the thread. I simply throw away the first (slow) poll and only use the second (fast) poll. Managed to get a ~4x speed improvement. Thank you for the help. Pseudo-code:

while (true)
   1: Do GET and trow away
   2: Do GET and use
   3: sleep for remainder of 1 sec
 end

Code adjustment:

//SLEEP every second time
if (i % 2 == 0 ){
    totalTimeFast += dur;
    System.out.println(i + ": " + statusLine + ", body length = "
        + body.length() + ", dur=" + dur + " ms   FAST");
    Thread.sleep(Math.max(0, 1000-dur));   
} else {
    totalTimeSlow += dur;
    System.out.println(i + ": " + statusLine + ", body length = "
        + body.length() + ", dur=" + dur + " ms   SLOW");
}   

And partial result of an N=100 test:

90: HTTP/1.1 200 OK, body length = 187136, dur=8 ms   FAST
91: HTTP/1.1 200 OK, body length = 186658, dur=51 ms   SLOW
92: HTTP/1.1 200 OK, body length = 186658, dur=34 ms   FAST
93: HTTP/1.1 200 OK, body length = 186461, dur=51 ms   SLOW
94: HTTP/1.1 200 OK, body length = 186461, dur=7 ms   FAST
95: HTTP/1.1 200 OK, body length = 186374, dur=42 ms   SLOW
96: HTTP/1.1 200 OK, body length = 186374, dur=7 ms   FAST
97: HTTP/1.1 200 OK, body length = 186612, dur=39 ms   SLOW
98: HTTP/1.1 200 OK, body length = 186612, dur=5 ms   FAST
99: HTTP/1.1 200 OK, body length = 186550, dur=36 ms   SLOW
100: HTTP/1.1 200 OK, body length = 186550, dur=13 ms   FAST
Got 100 results with a  totaltimeFast of 1541 ms and totalTimeSlow of 4503 ms