How to set a Host header value for SSL requests using HttpWebRequest

Question

I am trying to use the System.Net.HttpWebRequest class to perform a HTTP GET request to a specific web server for web applications we have load balanced over numerous servers. In order to achieve this, I need to be able to set the Host header value for the request, and I have been able to achieve this by using the System.Net.WebProxy class.

However, this all breaks down when I try to perform a GET using SSL. When I attempt to do this, the call to HttpWebRequest.GetResponse throws a System.Net.WebException, with a HTTP status code of 400 (Bad Request).

Is what I'm trying to achieve possible with HttpWebRequest, or should I be looking for an alternative way to perform what I want?

Here is the code I've been using to try and get this all to work :-

using System;
using System.Web;
using System.Net;
using System.IO;

namespace UrlPollTest
{
    class Program
    {
        private static int suffix = 1;
        static void Main(string[] args)
        {
            PerformRequest("http://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "");
            PerformRequest("https://www.microsoft.com/en/us/default.aspx", "www.microsoft.com");

            Console.WriteLine("Press any key to continue");
            Console.ReadKey();
        }

        static void PerformRequest(string AUrl, string AProxy)
        {
            Console.WriteLine("Fetching from {0}", AUrl);
            try
            {
                HttpWebRequest request = WebRequest.Create(AUrl) as HttpWebRequest;
                if (AProxy != "")
                {
                    Console.WriteLine("Proxy = {0}", AProxy);
                    request.Proxy = new WebProxy(AProxy);
                }

                WebResponse response = request.GetResponse();
                using (Stream httpStream = response.GetResponseStream())
                {
                    using (StreamReader reader = new StreamReader(httpStream))
                    {
                        string s = reader.ReadToEnd();
                        File.WriteAllText(string.Format("D:\\Temp\\Response{0}.html", suffix++), s);
                    }
                }
                Console.WriteLine("  Success");
            }
            catch (Exception e)
            {
                Console.WriteLine("  " + e.Message);
            }
        }
    }
}

Answer 1

I've used a hacky approach before - of having a test exe that modifies my "hosts" file, injecting an entry for the farm name to the specific server's address (and issuing an ipconfig /flushdns ). After that, requests should get routed to the right server as though it were the only one there.

Obviously this requires admin access... I use it as part of an automated smoke-test to hit the farm as though it were individual machines.

The other thing you might try is something on the NLB, perhaps in TCL (in the case of F5) - maybe add a custom header that the NLB understands? (assuming it is doing SSL re-signing).

Answer 2

In .NET 4.0 the Host header can be set independently of the URL that the request is going to so I would recommend that you use HttpwebRequest.Host to solve your problem. This (I think) will ship as part of .NET 4.0 beta2. There is NO way to re-write the host header to be different under previous versions of .Net without implementing your own custom subclass of WebRequest (believe me, it's been tried).