C# - Reading HTTP requests with StreamReader

Question

I am writing a TCP Client and a Server in C# which use manually written HTTP requests to communicate with each other. The trouble I have is with reading from the Network Stream using StreamReader . So far, I have tried many methods but to no avail.

The requests I get from my TCP Client are in in various forms. For updating database, the requests look like this ( CRLF is a constant I use to denote "\\r\\n" string):

HTTP 1.0:

"POST /" + name + " HTTP/1.0" + CRLF + "Content-Length: " + length + CRLF + CRLF + location;

HTTP 1.1:

"POST / HTTP/1.1" + CRLF + hostname + "Content-Length: " + length + CRLF + CRLF + nameLocString;

The requests are of correct form and the client is sending them correctly - I have tested this on a server to which I have access that responded to them without problems.

The problems I have are with my TCP Listener code. To avoid posting the whole code, I will just include the parts of the code that are problematic (found out by debugging).

Server code:

NetworkStream socketStream = new NetworkStream(connection);
StreamReader sr = new StreamReader(socketStream);

string input = ReadAllLinesWithNull(sr); // reading version 1
string input = ReadAllLinesWithEndOfStream(sr);  // reading version 2
string input = ReadAllLinesWithPeek(sr);  // reading version 3
string input = sr.ReadToEnd();  // reading version 4

And the methods used are:

static string ReadAllLinesWithNull(StreamReader sr)
{
    string input;
    string nextLine;
    input = sr.ReadLine();
    while ((nextLine = sr.ReadLine()) != null)
    {
        Console.WriteLine(input);
        input += nextLine;
    }
    sr.Close();
    return input;
}

static string ReadAllLinesWithEndOfStream(StreamReader sr)
{
    string input = "";
    while (!sr.EndOfStream)
    {
        input += sr.ReadLine();
    }
    sr.Close();
    return input;
}

static string ReadAllLinesWithPeek(StreamReader sr)
{
    string input = "";
    while (sr.Peek() >= 0)
    {
        input += sr.ReadLine();
    }
    sr.Close();
    return input;
}

None of these methods for reading worked. With my connection timeouts set, I have been getting IO Exception that it took too long to read/the connection was forcibly closed. I switched off the timeouts and the Read took indefinite amounts of time.

Thanks to using ReadLine() s I was able to single out the place where it ultimately hangs for all versions of protocol and found out that when there is cluster of two CRLFs ( "\\r\\n\\r\\n" ), the Stream Reader is not able to cope with this and gets stuck.

Do you have any suggestions as on how to get around this? I need to use the version with multiple CRLFs as it is in the specification.

If you need any additional information, I will try to supply it as sson as possible.

Answer 1

In the end I have found a solution to my problem. Instead of using

static string ReadAllLinesWithPeek(StreamReader sr)
{
    string input = "";
    while (sr.Peek() >= 0)
    {
        input += sr.ReadLine();
    }
    sr.Close();
    return input;
}

I had to use

static string ReadAllLinesWithPeek(StreamReader sr)
{
    string input = "";
    while (sr.Peek() >= 0)
    {
        input += (char) sr.Read();
    }
    return input;
}

I am still not sure why reading the input by lines did not work but when reading it by char at a time, it does.

Answer 2

A NetworkStream blocks on a Read operation if there is currently no data available and the other side has not closed that channel yet. TCP by itself has no concept of a message - that problem is to be solved at the HTTP level.

For HTTP you can keep reading until your data contains a \\r\\n\\r\\n sequence, which separates the header from the body. How to process the body depends on which headers are present:

• Transfer-Encoding: chunked indicates that the sender will send chunks of data and will end with a 0-length chunk
• Content-Length should be present when not using chunks, you can then read exactly that many bytes of data
• GET requests should not have a body, you can probably assume this if the above headers are not set
• Connection: close may be used for responses, indicating that the connection will be closed after all response data is sent

As you can see, StreamReader.ReadLine() would work pretty well on parsing the header, and it is quite suitable for reading chunks too, but it cannot be used for reading a fixed-length body.

I don't know how realiable it would be to read from a stream previously read from by a StreamReader (it can probably read ahead some data to its buffer), but slapping using blocks around them only causes the underlying stream to be closed unless you pick that one constructor overload .