How to read a HTTP response with gzip compressed?

Question

The following code is used to get a response from a website every five seconds, but there is always something wrong with the result according to the packet analyzer( such as wireshark ). I guess that maybe the HTTP response is compressed with GZIP and there may be something wrong when reading it. When I execute the code, I can get multiple Finishied in Eclipse( which means reading the response is finished ), but I can only send a few GET / 1.1 according to packet analyzer. Why? If I comment the code Thread.sleep(1000*5); , I will see Finished very quickly( too fast, which is abnormal according to my network speed ), but GET / 1.1 is still very slow. What's the matter with the code?

public class pediy {

    public static void main(String[] args) throws IOException {

        URL url = new URL("http://bbs.pediy.com");
        String request = "GET / HTTP/1.1\r\nHost: bbs.pediy.com\r\nProxy-Connection: keep-alive\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.56 Safari/537.17 CoolNovo/2.0.6.12\r\nAccept-Encoding: gzip,deflate,sdch\r\nAccept-Language: en-US;q=0.6,en;q=0.4\r\nAccept-Charset: utf-8;q=0.7,*;q=0.3\r\n";

        Socket socket = null;
        PrintWriter os = null;
        BufferedReader is = null;
        InputStreamReader isr = null;

        while(true) {

            socket = new Socket(url.getHost(), 80);
            os = new PrintWriter(socket.getOutputStream());
            isr = new InputStreamReader(socket.getInputStream());
            is=new BufferedReader(isr);

            try {
                while (true) {

                    os.println(request);
                    os.flush();


                    while(is.read() != -1);

                    System.out.println("Finished");

                    Thread.sleep(1000*5);
                }

            } catch (Exception e) {
                System.out.println("Error" + e);
            } finally {
                CloseAll(socket, os, is, isr);
            }
        }
    }

    private static void CloseAll(Socket socket, PrintWriter os, BufferedReader is, InputStreamReader isr) throws IOException {
        if(socket != null) socket.close();
        if(os != null) os.close();
        if(is != null) is.close();
        if(isr != null) isr.close();
    }
}

Answer 1

you didn't assign is. BufferedReader is = null; you need to assign is with socket.getInputStream()

 is = new BufferedReader(new InputStreamReader(socket.getInputStream())); 

Answer 2

One problem here is that the newline character in HTTP is \\r\\n , not just \\n . Another is that your request needs to end with a blank line after the headers. Another is that you are speaking HTTP 1.1 but you aren't sending a Content-Length. Another is that you are reading the stream to EOS and then expecting to be able to write another request and read another response. That cannot work: EOS means that the peer has closed the connection. Why aren't you using HttpURLConnection for this?