TcpClient's NetworkStream reads incomplete data unless Thread.Sleep(1) is called

Question

I have a method that reads some data through a TcpClient 's NetworkStream , and it has been giving me some errors.

During investigation, I discovered that it actually worked fine... but only if I stepped through the code with Visual Studio 2012's debugger, using a breakpoint.

Here's my code:

public static byte[] DownloadStream(string hostname, int port, 
    byte[] requestBytes, int bufferSize = 4096)
{
    byte[] responseBytes = null;

    var client = new System.Net.Sockets.TcpClient(hostname, port);

    if (client.Connected)
    {
        using (var stream = client.GetStream())
        {
            stream.Write(requestBytes, 0, requestBytes.Length);
            stream.Flush();

            if (stream.CanRead)
            {
                var responseStream = new System.IO.MemoryStream();

                byte[] buffer = new byte[bufferSize];
                int bytesRead = 0;

                do
                {
                    bytesRead = stream.Read(buffer, 0, buffer.Length);

                    responseStream.Write(buffer, 0, bytesRead);
                }
                while (stream.DataAvailable);

                responseBytes = responseStream.ToArray();
            }
        }
    }

    client.Close();

    return responseBytes;
}

This is pretty frustrating, since there's no real error. It apparently just needs the debugger to hold its hand while it reads the NetworkStream .

Does anybody know why this is happening? How can I fix it?

---

Edit:

For some reason making this change eliminates the problem:

                do
                {
                    bytesRead = stream.Read(buffer, 0, buffer.Length);

                    responseStream.Write(buffer, 0, bytesRead);

                     System.Threading.Thread.Sleep(1); //added this line
                }
                while (stream.DataAvailable);

Any insight on this?

Answer 1

The NetworkStream.DataAvailable property is not a reliable way to detect the end of the response; if the response is split into multiple TCP packets and a packet has not yet been delivered at the moment you check DataAvailable , then the property will return false, terminating your loop prematurely.

Apparently, your network connection is fast enough that Thread.Sleep(1) provides sufficient time for each successive packet to arrive. On a slower connection, though, it might not be adequate.

For reliable communication, the client and the server need to agree on a way to signal the end of the response. For example:

• The server can send the length of the response as a prefix before the actual response. The client reads the length, and then it reads from the stream until it has received that number of bytes.
• The server can send a special terminator as a suffix after the actual response. The client reads from the stream until it encounters the terminator. (Obviously, the terminator cannot appear anywhere in the response itself.)
• The server can close the connection after it sends the entire response. The client reads from the stream until the connection is closed.