HTTPClient, C# and XML Response

Question

I need to hit a REST service to obtain status information for some devices we have. This information is read-only.

I have successfully tested what I need to test using Fiddler, but unsure how to accomplish it via HTTPClient or some other routine? I'd like to simply write a C# Console Application.

Basically, the task is this:

1) Logging into REST service via login URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must send an XML payload of a user-defined XML doc. d) There will be an XML response with a session ID and set-cookie value that I will need to obtained.

2) I must obtain a device list via device list URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must pass Cookie header with the set-cookie value from initial response d) I must send an XML payload of a user-defined XML doc. Will also need to include session ID from initial response. e) There will be an XML response with a list of devices... There may be more than one with a unique device ID. I will need to parse out a particular device IDs.

3) I must obtain a device status via the device status URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must pass Cookie header with the set-cookie value from the initial response d) I must send and XML payload of a user-defined XML doc. Will also need to include session ID from initial response. e) There will be an XML response with a list of device status information. I will need to parse out particular status IDs.

If anyone can help me with this, it would be greatly appreciated.

I'm also wondering if I should create a Login, Device list, Device Status object with all the XML results and populate this into an object? Maybe this is overkill...

Answer 1

The key point here is in using the same CookieContainer for all requests in order to preserve the cookies:

var cookieContainer = new CookieContainer();

1) Logging into REST service via login URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must send an XML payload of a user-defined XML doc. d) There will be an XML response with a session ID and set-cookie value that I will need to obtain.

var request = (HttpWebRequest)WebRequest.Create("http://example.com/login");
request.CookieContainer = cookieContainer;
request.UserAgent = "Some User Agent";
request.ContentType = "text/xml";
request.Method = "POST";
using (var stream = request.GetRequestStream())
using (var writer = new StreamWriter(stream))
{
    writer.Write("<foo>bar</foo>");
}

using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
    string body = reader.ReadToEnd();
    // the body will probably contain the SessionId you are looking for
    // go ahead, parse it and extract the necessary information
}

2) I must obtain a device list via device list URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must pass Cookie header with the set-cookie value from initial response d) I must send an XML payload of a user-defined XML doc. Will also need to include session ID from initial response. e) There will be an XML response with a list of devices... There may be more than one with a unique device ID. I will need to parse out a particular device IDs.

var request = (HttpWebRequest)WebRequest.Create("http://example.com/devices");
request.CookieContainer = cookieContainer;
request.UserAgent = "Some User Agent";
request.ContentType = "text/xml";
request.Method = "POST";
using (var stream = request.GetRequestStream())
using (var writer = new StreamWriter(stream))
{
    writer.Write("<foo><baz>bazinga</baz><sessionid>... the session id you have read from the body of the previous request might come here ...</sessionid></foo>");
}

using (var response = request.GetResponse())
using (var stream = response.GetResponseStream())
using (var reader = new StreamReader(stream))
{
    string body = reader.ReadToEnd();
}

The important part is to reuse the same instance of the cookieContainer for both requests. Bear in mind that this will only work of both requests are to the same domain which I suppose is your case because you appear to be scraping a website, not consuming a REST API. A REST API would never rely on cookies by the way. But if for some reason yours does, and is cross domain then you will have to extract the cookies from the cookieContainer of the request and manually attach the value to the second request as the Cookie header:

request.Headers[HttpRequestHeader.Cookie] = "Value of the cookie you have read from the cookieContainer of the first request";

3) I must obtain a device status via the device status URL using POST a) I must pass a User-Agent header matching a user-defined string b) I must pass Content-Type header matching text/xml c) I must pass Cookie header with the set-cookie value from the initial response d) I must send and XML payload of a user-defined XML doc. Will also need to include session ID from initial response. e) There will be an XML response with a list of device status information. I will need to parse out particular status IDs.

You should already have all the necessary bits to implement this yourself.