HttpWebRequest如何处理（过早）关闭底层TCP连接？

Question

I have a hard time figuring out if there is a way to handle potential connectivity problems when using .NET's HttpWebRequest class to call a remote server (specifically a REST web service). 在使用.NET的HttpWebRequest类调用远程服务器（特别是REST Web服务）时，我很难搞清楚是否有办法处理潜在的连接问题。 From my investigations the behaviour of the WebClient class is the same, which is somewhat expected since it appears to only offer a more simple interface to the HttpWebRequest. 从我的调查来看，WebClient类的行为是相同的，这是有点预期的，因为它似乎只提供了一个更简单的HttpWebRequest接口。

For simulation purposes, I've written a very simple HTTP server that does not behave according to the HTTP 1.1 RFC. 出于模拟目的，我编写了一个非常简单的HTTP服务器，它不符合HTTP 1.1 RFC。 What it does is it accepts a client connection, then sends appropriate HTTP 1.1 headers and a "Hello World!" 它的作用是接受客户端连接，然后发送适当的HTTP 1.1头和“Hello World！”。 payload back to the client and closes the socket, the thread accepting client connections on the server side looks as follows: 有效负载返回客户端并关闭套接字，服务器端接受客户端连接的线程如下所示：

    private const string m_defaultResponse = "<html><body><h1>Hello World!</h1></body></html>";
    private void Listen()
    {
        while (true)
        {
            using (TcpClient clientConnection = m_listener.AcceptTcpClient())
            {
                NetworkStream stream = clientConnection.GetStream();
                StringBuilder httpData = new StringBuilder("HTTP/1.1 200 OK\r\nServer: ivy\r\nContent-Type: text/html\r\n");
                httpData.AppendFormat("Content-Length: {0}\r\n\r\n", m_defaultResponse.Length);
                httpData.AppendFormat(m_defaultResponse);

                Thread.Sleep(3000); // Sleep to simulate latency

                stream.Write(Encoding.ASCII.GetBytes(httpData.ToString()), 0, httpData.Length);

                stream.Close();

                clientConnection.Close();
            }
        }
    }

Since the HTTP 1.1 RFC states that HTTP 1.1 by default keeps connections alive and that a server must send a "Connection: Close" response header if it wants to close a connection this is unexpected behaviour for the client-side. 由于HTTP 1.1 RFC声明HTTP 1.1默认保持连接活动，并且服务器必须发送“Connection：Close”响应头，如果它想要关闭连接，这是客户端的意外行为。 The client uses HttpWebRequest in the following way: 客户端以下列方式使用HttpWebRequest：

    private static void SendRequest(object _state)
    {
        WebResponse resp = null;

        try
        {
            HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://192.168.0.32:7070/asdasd");
            request.Timeout = 50 * 1000;

            DateTime requestStart = DateTime.Now;
            resp = request.GetResponse();
            TimeSpan requestDuration = DateTime.Now - requestStart;

            Console.WriteLine("OK. Request took: " + (int)requestDuration.TotalMilliseconds + " ms.");
        }
        catch (WebException ex)
        {
            if (ex.Status == WebExceptionStatus.Timeout)
            {
                Console.WriteLine("Timeout occurred");
            }
            else
            {
                Console.WriteLine(ex);
            }
        }
        finally
        {
            if (resp != null)
            {
                resp.Close();
            }

            ((ManualResetEvent)_state).Set();
        }
    }

The above method is queued via ThreadPool.QueueUserWorkItem(waitCallback, stateObject). 上述方法通过ThreadPool.QueueUserWorkItem（waitCallback，stateObject）排队。 The ManualResetEvent is used to control queuing behavior so that not the entire thread pool gets filled up with waiting tasks (since the HttpWebRequest implicitly uses worker threads because it functions asynchronously internally to implement the timeout functionality). ManualResetEvent用于控制排队行为，以便整个线程池不会被等待任务填满（因为HttpWebRequest隐式使用工作线程，因为它在内部异步运行以实现超时功能）。

The problem with all this is that once all connections of the HttpWebRequest's underlying ServicePoint are "used up" (ie closed by the remote server), there will be no new ones opened up. 所有这一切的问题是，一旦HttpWebRequest的底层ServicePoint的所有连接都“用完”（即由远程服务器关闭），就不会打开新的连接。 It also does not matter if the ConnectionLeaseTimeout of the ServicePoint is set to a low value (10 seconds). 如果ServicePoint的ConnectionLeaseTimeout设置为低值（10秒）也无关紧要。 Once the system gets into this state, it will no longer function properly because it does not reconnect automatically and all subsequent HttpWebRequests will time out. 一旦系统进入此状态，它将不再正常运行，因为它不会自动重新连接，并且所有后续HttpWebRequests都将超时。 Now the question really is if there is a way to solve this by somehow destroying a ServicePoint under certain conditions or closing underlying connections (I did not have any luck with ServicePoint.CloseConnectionGroup() yet, the method is also pretty undocumented in terms of how to properly use it). 现在的问题是，如果有办法通过某种方式在某些条件下销毁ServicePoint或关闭底层连接来解决这个问题（我对ServicePoint.CloseConnectionGroup（）没有任何好运，那么该方法在如何解释方面也很简单正确使用它）。

Does anybody have any idea how I could approach this problem? 有谁知道我怎么能解决这个问题？

Answer 1

The solution I came up with based on some of the ideas here is to manage the connections myself. 我根据这里的一些想法提出的解决方案是自己管理连接。 If a unique ConnectionGroupName is assigned to a WebRequest (eg Guid.NewGuid().ToString()), a new connection group with one connection will be created in the ServicePoint for the request. 如果将唯一的ConnectionGroupName分配给WebRequest（例如Guid.NewGuid（）。ToString（）），则将在ServicePoint中为请求创建具有一个连接的新连接组。 Note that there's no more connection limiting at this point, since .NET limits per connection group rather than per ServicePoint, so you'll have to handle it yourself. 请注意，此时不再有连接限制，因为.NET限制每个连接组而不是每个ServicePoint，因此您必须自己处理它。 You'll want to reuse connection groups so that existing connections with KeepAlive are reused, but if a WebException exception occurs, the request's connection group should be destroyed since it might be stale. 您将需要重用连接组，以便重用与KeepAlive的现有连接，但是如果发生WebException异常，则应该销毁请求的连接组，因为它可能是陈旧的。 Something like this (create a new instance for each host name): 这样的事情（为每个主机名创建一个新实例）：

public class ConnectionManager {
    private const int _maxConnections = 4;

    private Semaphore _semaphore = new Semaphore(_maxConnections, _maxConnections);
    private Stack<string> _groupNames = new Stack<string>();

    public string ObtainConnectionGroupName() {
        _semaphore.WaitOne();
        return GetConnectionGroupName();
    }

    public void ReleaseConnectionGroupName(string name) {
        lock (_groupNames) {
            _groupNames.Push(name);
        }
        _semaphore.Release();
    }

    public string SwapForFreshConnection(string name, Uri uri) {
        ServicePoint servicePoint = ServicePointManager.FindServicePoint(uri);
        servicePoint.CloseConnectionGroup(name);
        return GetConnectionGroupName();
    }

    private string GetConnectionGroupName() {
        lock (_groupNames) {
            return _groupNames.Count != 0 ? _groupNames.Pop() : Guid.NewGuid().ToString();
        }
    }
}

Answer 2

This is a horrible hack, but it works. 这是一个可怕的黑客，但它的确有效。 Call it periodically if you notice your connections are getting stuck. 如果您发现连接卡住了，请定期打电话。

    static public void SetIdle(object request)
    {
        MethodInfo getConnectionGroupLine = request.GetType().GetMethod("GetConnectionGroupLine", BindingFlags.Instance | BindingFlags.NonPublic);
        string connectionName = (string)getConnectionGroupLine.Invoke(request, null);

        ServicePoint servicePoint = ((HttpWebRequest)request).ServicePoint;
        MethodInfo findConnectionGroup = servicePoint.GetType().GetMethod("FindConnectionGroup", BindingFlags.Instance | BindingFlags.NonPublic);
        object connectionGroup;
        lock (servicePoint)
        {
            connectionGroup = findConnectionGroup.Invoke(servicePoint, new object[] { connectionName, false });
        }

        PropertyInfo currentConnections = connectionGroup.GetType().GetProperty("CurrentConnections", BindingFlags.Instance | BindingFlags.NonPublic);
        PropertyInfo connectionLimit = connectionGroup.GetType().GetProperty("ConnectionLimit", BindingFlags.Instance | BindingFlags.NonPublic);

        MethodInfo disableKeepAliveOnConnections = connectionGroup.GetType().GetMethod("DisableKeepAliveOnConnections", BindingFlags.Instance | BindingFlags.NonPublic);

        if (((int)currentConnections.GetValue(connectionGroup, null)) ==
            ((int)connectionLimit.GetValue(connectionGroup, null)))
        {
            disableKeepAliveOnConnections.Invoke(connectionGroup, null);
        }

        MethodInfo connectionGoneIdle = connectionGroup.GetType().GetMethod("ConnectionGoneIdle", BindingFlags.Instance | BindingFlags.NonPublic);
        connectionGoneIdle.Invoke(connectionGroup, null);
    }

Answer 3

Here is my suggestion. 这是我的建议。 I have not tested it. 我没有测试过。 Alter reference.cs 改变reference.cs

    protected override WebResponse GetWebResponse(WebRequest request)
    {
        try
        {
            return base.GetWebResponse(request);
        }
        catch (WebException)
        {
            HttpWebRequest httpWebRequest = request as HttpWebRequest;
            if (httpWebRequest != null && httpWebRequest.ServicePoint != null)
                httpWebRequest.ServicePoint.CloseConnectionGroup(httpWebRequest.ConnectionGroupName);

            throw;
        }
    }

HttpWebRequest如何处理（过早）关闭底层TCP连接？

问题描述

3 个解决方案

解决方案1
6 2010-12-15 03:12:51

解决方案2
2 2009-11-11 09:46:10

解决方案3
1 2010-08-09 02:13:32

HttpWebRequest如何处理（过早）关闭底层TCP连接？

问题描述

3 个解决方案

解决方案1 6 2010-12-15 03:12:51

解决方案2 2 2009-11-11 09:46:10

解决方案3 1 2010-08-09 02:13:32

解决方案1
6 2010-12-15 03:12:51

解决方案2
2 2009-11-11 09:46:10

解决方案3
1 2010-08-09 02:13:32