Debugging an HttpWebResponse

Question

This is being done on a Windows Forms App. I've spent a ton of time stepping through this code with the debugger. What I've found are the following things and they all seem to be at this line:

using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())

1. If I include

request.SendChunked = true;

I get this error at the response line previously stated:

'System.Net.WebException: The remote server returned an error: (415) Unsupported Media Type.

2. If I comment out the code in #1, I receive this error at that main response line that I mentioned in the beginning:

'System.Net.WebException: The underlying connection was closed: The connection was closed unexpectedly.

3. If I go with route #1, the "Connection" of the request remains as "KeepAlive" all the way through. But if I go with route #2, the "Connection" of the request changes to "null" at the response line that I mentioned in the beginning.

    private void HttpPost()
    {


        HttpWebRequest request = null;
        Uri uri = new Uri("https://post.craigslist.org/bulk-rss/post");

        request = (HttpWebRequest)WebRequest.Create(uri);
        request.Method = "POST";
        request.ContentType = "application/x-www-form-urlencoded";
        XmlDocument doc = new XmlDocument();
        doc.Load("XMLFile1.xml");
        //request.ContentLength = doc.InnerXml.Length;

        request.SendChunked = true;
        using (Stream writeStream = request.GetRequestStream())
        {
            UTF8Encoding encoding = new UTF8Encoding();
            byte[] bytes = encoding.GetBytes(doc.InnerXml);
            //request.ContentLength = bytes.Length;
            writeStream.Write(bytes, 0, bytes.Length);
        }


        string result = string.Empty;

        request.ProtocolVersion = System.Net.HttpVersion.Version11;
        request.KeepAlive = false;
        try
        {
            using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
            {
                using (Stream responseStream = response.GetResponseStream())
                {
                    using (System.IO.StreamReader readStream = new System.IO.StreamReader(responseStream, Encoding.UTF8))
                    {
                        result = readStream.ReadToEnd();
                    }
                }

            }

        }
        catch (Exception e)
        {
            string innerException = String.Format("Inner exception: '{0}'", e.Data);
            string exceptionCause = String.Format("An error occurred: '{0}'", e);
            System.IO.File.WriteAllText(@"C:\Users\Nathan\Documents\DebugOutputFile\exception.txt", exceptionCause);
            System.IO.File.WriteAllText(@"C:\Users\Nathan\Documents\DebugOutputFile\innerException.txt", innerException);

        }
    }

I feel like these things are adding up towards a solution, but I could really use some guidance.

Answer 1

Option 1: Change your content type to match the body encoding

request.ContentType = "application/xml";

Option 2: Change your body encoding to match the specified content-type

If your server expects only "application/x-www-form-urlencoded", then you need to change your body encoding to suit it, for example, like this:

    using (Stream writeStream = request.GetRequestStream())
    {
        UTF8Encoding encoding = new UTF8Encoding();
        string response = String.Concat("arg=", HttpUtility.UrlEncode(doc.InnerXml))
        byte[] bytes = encoding.GetBytes(doc.InnerXml);
        //request.ContentLength = bytes.Length;
        writeStream.Write(bytes, 0, bytes.Length);
    }

You need to know the parameter name (above was set to "arg") and add a reference to System.Web, if you haven't.

See following XML...

<?xml version="1.0" encoding="UTF-8"?><test></test>

and encoded string for reference (your request body should look similar to this):

arg=%3C%3Fxml%20version%3D%221.0%22%20encoding%3D%22UTF-8%22%3F%3E%3Ctest%3E%3C%2Ftest%3E

Explanation

If you look at the response you are getting with the first approach: 415 - Unsupported Media Type , you can notice that the content type you are specifying ( "application/x-www-form-urlencoded" ) doesn't match what you are sending in the body (an XML document). Chunk encoding should be enabled when sending files.

Note

When you are having trouble with a request done in source code, try to test the request alone with a web debugging tool, like Fiddler . There you would compose and issue the request until you get the response you want. Then you can compare that with what you are sending from source code (again you should use the same tool for inspecting your request).