How to send lower-case Keep-Alive header through HttpWebRequest

Question

I'm writing a bot, which should emulate firefox as closely as possible. By examining the headers that it is sending, I've found one minor difference, that I do not know how to get rid off:

Firefox uses following keep-alive header:

Connection: keep-alive

While c# always sends out:

Connection: Keep-Alive

I know that it probably does not matter, but I'd still love to know if there is any way/hack to modify that header to be all lower case.

Any ideas how to do this?

Answer 1

In .net 4.0 this works:

request.Headers.GetType().InvokeMember(
    "ChangeInternal",
    BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.InvokeMethod,
    Type.DefaultBinder, 
    request.Headers, 
    new object[] { "Connection", "keep-alive" }
);

Just be sure you don't actually set the KeepAlive property on the request

Answer 2

I also need to do this using C# and have been trying with HttpWebRequest. A non .NET SOAP web service I want to call expects Connection: keep-alive in lower case too.

I would rather not go down the route of doing socket programming for this, so if you have any suggestions on how you worked around this, if you did it would be very helpful.

My investigation so far :

When using http Protocol version 1.1, the header isn't even sent, even if you specify the property.

eg

var request = (HttpWebRequest)WebRequest.Create(Endpoint); 
request.KeepAlive = true;

The solution for this is to use System.Reflection to modify the httpBehaviour which will mean Keep-Alive is sent. This will send an initial upper-case K and A 'Keep-Alive' on every request.

var sp = request.ServicePoint; 
var prop = sp.GetType().GetProperty("HttpBehaviour", BindingFlags.Instance | BindingFlags.NonPublic);
prop.SetValue(sp, (byte)0, null);

I tried using System.Reflection to modify the header too. The below code will add the flag correctly in lower case :

request.Headers.GetType().InvokeMember("ChangeInternal", BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.InvokeMethod, Type.DefaultBinder, request.Headers, new object[] { "Connection", "keep-alive" });

However, at the point where I am calling GetResponse

var response = (HttpWebResponse)request.GetResponse()

The header is wiped out. Looking at the source code for HttpWebRequest.cs, this happens right before sending headers to the wire.

    //
    // The method is called right before sending headers to the wire*
    // The result is updated internal _WriteBuffer
    //
    // See ClearRequestForResubmit() for the matching cleanup code path.
    // 
    internal void SerializeHeaders() {

    ....
    ....

        string connectionString = HttpKnownHeaderNames.Connection; 
        if (UsesProxySemantics || IsTunnelRequest ) { 
            _HttpRequestHeaders.RemoveInternal(HttpKnownHeaderNames.Connection);
            connectionString = HttpKnownHeaderNames.ProxyConnection; 
            if (!ValidationHelper.IsBlankString(Connection)) {
                _HttpRequestHeaders.AddInternal(HttpKnownHeaderNames.ProxyConnection, _HttpRequestHeaders[HttpKnownHeaderNames.Connection]);
            }
        } 
        else {
            _HttpRequestHeaders.RemoveInternal(HttpKnownHeaderNames.ProxyConnection); 
        } 

RemoveInternal will also remove the header we have hacked in using Reflection.

So this still leaves me stuck.

Are there other any way around this other than going at a socket level? Are there other classes or 3rd party libraries which allow me to amend headers as I wish?

Sorry this is not an answer, but I can't yet comment on your question.

Answer 3

Connection: keep-alive is default header of Chrome and Firefox browser.

Connection: Keep-Alive is default header of Internet Explorer. Absolutely

Connection: Keep-Alive is default header of HttpWebRequest. I think you should writing a bot like IE is best choice if using HttpWebRequest.

Answer 4

Using reflection, you can replace a WebHeaderCollection 's internal NameValueCollection with a custom implementation, such as the following:

// When injected into a WebHeaderCollection, ensures that
// there's always exactly one "Connection: keep-alive" header.
public class CustomNameValueCollection : NameValueCollection {

    private const BindingFlags allInstance =
        BindingFlags.Instance | BindingFlags.Public | BindingFlags.NonPublic;

    private static readonly PropertyInfo innerCollProperty =
        typeof(WebHeaderCollection).GetProperty("InnerCollection", allInstance);

    private static readonly FieldInfo innerCollField =
        typeof(WebHeaderCollection).GetField("m_InnerCollection", allInstance);

    public static void InjectInto(WebHeaderCollection coll) {
        // WebHeaderCollection uses a custom IEqualityComparer for its internal
        // NameValueCollection. Here we get the InnerCollection property so that
        // we can reuse its IEqualityComparer (via our constructor).
        var innerColl = (NameValueCollection) innerCollProperty.GetValue(coll);
        innerCollField.SetValue(coll, new CustomNameValueCollection(innerColl));
    }

    private CustomNameValueCollection(NameValueCollection coll) : base(coll) {
        Remove("Connection");
        base.Add("Connection", "keep-alive");
    }

    public override void Add(string name, string value) {
        if (name == "Connection") return;
        base.Add(name, value);
    }
}

Use it like this:

var request = (HttpWebRequest) WebRequest.Create("https://www.google.com/");
CustomNameValueCollection.InjectInto(request.Headers);
using (var response = request.GetResponse()) {
    ...
}