Reading from a NetworkStream fails after StreamReader did

Question

I have a method that reads an HTTP request from a Stream (in fact it's a NetworkStream) in a very primitive way. First I read the header line by line and when I see the separator (aka empty line) I start reading the body.

The code looks like this:

public byte[] ReceiveHttp(Stream stream)
{
    var headerLines = new List<string>();
    using (var reader = new StreamReader(stream, Encoding.ASCII, false, 8192, true))
    {
        var requestLine = reader.ReadLine();
        while (true)
        {
            var headerLine = reader.ReadLine();
            if (string.IsNullOrEmpty(headerLine))
            {
                // all header lines read
                break;
            }
            headerLine = headerLine.Trim();
            headerLines.Add(headerLine);
        }
    }

    var contentLength = RetrieveContentLengthFromHeaderLines(headerLines);

    var bytes = new byte[contentLength];
    var readbytesCount = 0;

    while (readbytesCount < contentLength)
    {
        var chunkBytesCount = stream.Read(bytes, 0, contentLength);
        readbytesCount += chunkBytesCount;
    }

    return bytes;
}

Now my problem is, that the subsequent read of the body keeps hanging sometimes. I mean the call to stream.Read blocks until some kind of timeout is reached.

When replacing all StreamReader.ReadLine() (in fact removing StreamReader from the code) with some custom code, that reads a line byte by byte until the line break occurs, the problem never occurs.

So, I guess there is some kind of buffering/caching inside StreamReader that may already have read the whole message but hides it from subsequent stream actions. But that's only a guess.

Does somebody have an idea, what causes this behavior and how I could get around it without using a custom byte-by-byte ReadLine method?

Answer 1

I have already commented that it is quite complex (read impossible) to use StreamReader to read the headers of the http protocol. You have to read the Stream directly and split "by hand".

My line splitter:

// If you want to use UTF8:
// var encoding = (Encoding)Encoding.UTF8.Clone();
var encoding = Encoding.GetEncoding("iso-8859-1"); 
var decoder = encoding.GetDecoder();

//Encoding.ASCII
var headerLines = new List<string>();

var sb = new StringBuilder();
byte[] bytes = new byte[1];
char[] chars = new char[2];

while (true)
{
    int curr = stream.ReadByte();
    char ch = '\0';

    bool newLine = false;

    if (curr == -1)
    {
        newLine = true;
    }
    else
    {
        bytes[0] = (byte)curr;

        // There is the possibility of a partial invalid 
        // character (first byte of UTF8) plus a new valid 
        // character. In this case decoder.GetChars will
        // return 2 chars
        int count = decoder.GetChars(bytes, 0, 1, chars, 0);

        for (int i = 0; i < count; i++)
        {
            ch = chars[i];

            if (ch == '\n')
            {
                newLine = true;
            }
            else
            {
                sb.Append(ch);
            }
        }
    }

    if (newLine)
    {
        string str = sb.ToString();

        // Handling of \r\n
        if (ch == '\n' && str[str.Length - 1] == '\r')
        {
            str = str.Remove(str.Length - 1);
        }

        str = str.Trim();

        if (str.Length != 0)
        {
            headerLines.Add(str);
            sb.Clear();
        }
        else
        {
            break;
        }
    }

    if (curr == -1)
    {
        break;
    }
}

Note that for headers the suggested encoding is ISO-8859-1. The line ending can be both \\r\\n and \\n . The code is quite complex because I wanted to handle UTF8. For other encodings it should be useless. Note that I Trim() before doing the check for length, so a line full only of spaces will still stop the reading of headers.