性能问题。 使用java从套接字读取字节时速度太慢

Question

these days I'm confused about the Tcp performance while using java socket. In fact the java code is very simple. details as below:

1. server open a port and begin to listen.
2. client request and after connect to server, client begin to write to socket.
3. after server got the request, it will open a new thread to handle this connection. (this connection is a long connection which will not time out).
4. the server will keep reading until it got the end separator, then give a response to the client and continue to keep reading again.
5. after client get the response, it will send another request again.

I find if the client write the whole message (including the end separator) one time, the communication speed is good satisfactorily, the speed can reach to 50000 messages per minute. How ever, if the client write the bytes to socket in separated times, the speed cut down quickly, just almost 1400 messages per minute, it is 1/40 times compared with the original speed. I'm quite confused about it. Any one could give me a hand? Any comments is appreciated!

my simulated server side is as below:

public class ServerForHelp {

    final static int BUFSIZE = 10240;
    Socket socket;
    String delimiter = "" + (char) 28 + (char) 13;

    public static void main(String[] args) throws IOException {

            ServerSocket ss = new ServerSocket(9200);
            System.out.println("begin to accept...");
            while (true) {
                Socket s = ss.accept();
                Thread t = new Thread(new SocketThread1(s));
                t.start();
            }
    }

    public String readUntilDelimiter() throws Exception {
        StringBuffer stringBuf = new StringBuffer();
        InputStream stream = socket.getInputStream();
        InputStreamReader reader = null;
        reader = new InputStreamReader(stream);

        char[] buf = new char[BUFSIZE];

        while (true) {
            int n = -1;
                n = reader.read(buf, 0, BUFSIZE);
            if (n == -1) {
                return null;  // it means the client has closed the connection, so return null.
            } else if (n == 0) {
                continue; // continue to read the data until got the delimiter from the socket.
            }

            stringBuf.append(buf, 0, n);
            String s = stringBuf.toString();

            int delimPos = s.indexOf(delimiter);
            if (delimPos >= 0) {
                // found the delimiter; return prefix of s up to separator and
                // To make the thing simple, I have discarded the content after the delimiter.
                String result = s.substring(0, delimPos);
                sendTheResponse(socket);
                return result;
            }
        }
    }

    private void sendTheResponse(Socket socket) throws IOException {
        Writer writer = new BufferedWriter(new OutputStreamWriter(socket.getOutputStream()));
        writer.write("Hi, From server response");
        writer.flush();
    }

}

class SocketThread1 implements Runnable {

    Socket socket;

    public SocketThread1(Socket socket) {
        this.socket = socket;
    }

    @Override
    public void run() {
        ServerForHelp server = new ServerForHelp();
        server.socket = socket;
        while (true) {
            try {
                if (server.readUntilDelimiter() == null) // it means that the client has closed the connection, exist
                    break;
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
    }

}

It is a normal socket programming.

and the following is my client side:

public void execute() throws Exception{

        int msgCnt = 0;
        Socket socket = null;
        byte[] bufBytes = new byte[512];
        long start = 0;
        final char START_MESSAGE = 0x0B;
        final char END_MESSAGE = 0x1C;
        final char END_OF_RECORD = 0x0D;//\r
        String MESSAGE = "HELLO, TEST";
        socket = new Socket("192.168.81.39", 9200);
        OutputStream os = socket.getOutputStream();
        InputStream is = socket.getInputStream();

        while (System.currentTimeMillis() - start < 60000)
        {

            // If you send the total message at one time, the speed will be improved significantly  

            // FORMAT 1
            StringBuffer buf = new StringBuffer();
            buf.append(START_MESSAGE);
            buf.append(MESSAGE);
            buf.append(END_MESSAGE);
            buf.append(END_OF_RECORD);
            os.write(buf.toString().getBytes());
            // FORMAT 1 END

            //FORMAT 2
//        os.write(START_MESSAGE);
//        os.write(MESSAGES[port].getBytes());
//        os.write(END_MESSAGE);
//        os.write(END_OF_RECORD);
            //FORMAT 2 END
            os.flush();
            is.read(bufBytes);
            msgCnt++;

            System.out.println(msgCnt);
        }
        System.out.println( msgCnt + " messages per minute");
    }

If I use the "FORMAT 1", to send the message, the speed could reach to 50000 messages per minute, but If use "FORMAT 2", the speed is down to 1400 messages per minute. Who is clear about the reason?

I'm trying to describe as detail as I can and any help will be appreciated very much.

Answer 1

Multiple very short writes to a socket in rapid succession followed by a read can trigger a bad interaction between Nagle's algorithm and TCP delayed acknowledgment ; even if you disable Nagle's algorithm, you'll cause an entire packet to be sent per individual write call (with 40+ bytes of overhead, whether the write is one byte or a thousand).

Wrapping a BufferedOutputStream around the socket's output stream should give you performance similar to "FORMAT 1" (precisely because it holds things in a byte array until it fills or is flushed).

As John Nagle explained on Slashdot :

The user-level solution is to avoid write-write-read sequences on sockets. write-read-write-read is fine. write-write-write is fine. But write-write-read is a killer. So, if you can, buffer up your little writes to TCP and send them all at once.