Why does HttpWebResponse return a null terminated string?

Question

I recently was using HttpWebResponse to return xml data from a HttpWebRequest, and I noticed that the stream returned a null terminated string to me.

I assume its because the underlying library has to be compatible with C++, but I wasn't able to find a resource to provide further illumination.

Mostly I'm wondering if there is an easy way to disable this behavior so I don't have to sanitize strings I'm passing into my xml reader.

---

Edit here is a sample of the relevant code:

httpResponse.GetResponseStream().Read(serverBuffer, 0, BUFFER_SIZE);
output = processResponse(System.Text.UTF8Encoding.UTF8.GetString(serverBuffer))

where processResponse looks like:

 processResponse(string xmlResponse)
 {
     var Parser = new XmlDocument();
     xmlResponse = xmlResponse.Replace('\0',' '); //fix for httpwebrequest null terminating strings
     Parser.LoadXml(xmlResponse);

Answer 1

This definitely isn't normal behaviour. Two options:

• You made a mistake in the reading code (eg creating a buffer and then calling Read on a stream, expecting it to fill the buffer)
• The web server actually returned a null-terminated response

You should be able to tell the difference using Wireshark if nothing else.

Answer 2

Hmm... I doubt it returns a null-terminated string since simply there is no such concept in C#. At best you could have a string with a \\0u0000 character at the end, but in this case it would mean that the return from the server contains such a character and the HttpWebRequest is simply doing it's duty and returns whatever the server returned.

Update

after reading your code, the mistake is pretty obvious: you are Read() -ing from a stream into a byte[] but not tacking notice of how much you actually read :

int responseLength = httpResponse.GetResponseStream().Read(
    serverBuffer, 0, BUFFER_SIZE);
output = processResponse(System.Text.UTF8Encoding.UTF8.GetString(
    serverBuffer, 0, responseLength));

this would fix the immediate problem, leaving only the other bugs in your code to deal with, like the fact that you cannot handle correctly a response larger than BUFFER_SIZE ... I would suggest you open a XML document reader on the returned stream instead of manipulating the stream via an (unnecessary) byte[ ] copy operation:

Parser.Load(httpResponse.GetResponseStream());

Answer 3

Could it be that you are setting a size (wrong size) to the buffer you are loading?

You can use a StreamReader to avoid the temp buffer if you don't need it.

using(var stream = new StreamReader(httpResponse.GetResponseStream()))
{
  string output = stream.ReadToEnd();
//...
}