多線程應用中的HttpWebRequest.GetResponse性能問題

Question

當從不同的線程為不同的URL調用HttpWebRequest.GetResponse方法時，我的性能非常糟糕。 例如，如果我們有一個線程並執行url1，則需要3秒。 如果我們在parallet中執行url1和url2，則需要10秒，第一個請求在8秒后結束，第二個請求在10秒后結束。

如果我們排除10個網址url1，url2，... url0，則需要1分4秒！ 第一個請求在50秒后結束！

我使用GetResponse方法。 我嘗試設置DefaultConnectionLimit但沒有幫助。 如果使用BeginGetRequest / EndGetResponse方法，則它的運行速度非常快，但前提是此方法是從一個線程調用的。 如果從不同的角度來看也很慢。 我需要一次從多個線程執行Http請求。

在每個線程中執行相同的代碼。 如果只有一個線程，則GetResponse方法的運行速度非常快。 URL的IP地址對於每個線程也不同。 如果編譯並運行以下代碼，您將看到請求被一個一個地處理。 拳頭執行3秒，第二秒8秒，第三秒15秒..即多線程沒有收益。

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.IO;
using System.Xml;
using System.Reflection;
using System.Threading;
using System.Collections.Generic;
using System.Collections.Specialized;
using System.Collections;

namespace HttpRequestExample
{

    class HttpPerformer
    {
        private Thread thread = null;
        HttpWebRequest httpRequest = null;

        public void start(string url)
        {
            thread = new Thread(new ThreadStart(WorkerThread));
            thread.Name = url;
            thread.Start();

        }

        public void WorkerThread()
        {
            try
            {
                HttpWebRequest req = (HttpWebRequest)WebRequest.Create((string)thread.Name);
                Console.WriteLine(DateTime.Now + " : before get response " + thread.Name);
                HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
                Console.WriteLine(DateTime.Now + " : after get response " + thread.Name);

            }
            catch (Exception e)
            {
                Console.WriteLine(DateTime.Now + " : Exception : " + e.Message + thread.Name);
            }
        }
    }

    class HttpAccessUtils
    {
        public static bool SetAllowUnsafeHeaderParsing20()
        {
            //Get the assembly that contains the internal class 
            Assembly aNetAssembly = Assembly.GetAssembly(typeof(System.Net.Configuration.SettingsSection));
            if (aNetAssembly != null)
            {
                //Use the assembly in order to get the internal type for the internal class 
                Type aSettingsType = aNetAssembly.GetType("System.Net.Configuration.SettingsSectionInternal");
                if (aSettingsType != null)
                {
                    //Use the internal static property to get an instance of the internal settings class. 
                    //If the static instance isn't created allready the property will create it for us. 
                    object anInstance = aSettingsType.InvokeMember("Section",
                    BindingFlags.Static | BindingFlags.GetProperty | BindingFlags.NonPublic, null, null, new object[] { });

                    if (anInstance != null)
                    {
                        //Locate the private bool field that tells the framework is unsafe header parsing should be allowed or not 
                        FieldInfo aUseUnsafeHeaderParsing = aSettingsType.GetField("useUnsafeHeaderParsing", BindingFlags.NonPublic | BindingFlags.Instance);
                        if (aUseUnsafeHeaderParsing != null)
                        {
                            aUseUnsafeHeaderParsing.SetValue(anInstance, true);
                            return true;
                        }
                    }
                }
            }
            return false;
        }
    }

    class Program
    {

        static void Main(string[] args)
        {
            HttpAccessUtils.SetAllowUnsafeHeaderParsing20();
            ServicePointManager.UseNagleAlgorithm = true;
            ServicePointManager.Expect100Continue = true;
            ServicePointManager.CheckCertificateRevocationList = true;
            ServicePointManager.DefaultConnectionLimit = 200; //ServicePointManager.DefaultPersistentConnectionLimit;
            ServicePointManager.MaxServicePoints = 100;
            Console.WriteLine(ServicePointManager.MaxServicePoints);


            ArrayList a = new ArrayList(150);

            for (int i = 100; i < 220; i++)
            {
                a.Add("http://207.242.7." + i.ToString());
            }

            for (int i = 0; i < a.Count; i++)
            {
                HttpPerformer hp = new HttpPerformer();
                hp.start((string)a[i]);
            }

        }

        static void performRequest(object url)
        {
            try
            {
                HttpWebRequest req = (HttpWebRequest)WebRequest.Create((string)url);
                Console.WriteLine(DateTime.Now + " : before get response " + url);
                HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
                Console.WriteLine(DateTime.Now + " : after get response " + url);
            }
            catch (Exception e)
            {
                Console.WriteLine(DateTime.Now + " : Exception : " + e.Message + (string)url);
            }

        }
    }
}

有人遇到過這樣的問題嗎？ 感謝您的任何建議。

Answer 1

獲取響應流后，您需要關閉conn ，否則連接將保持打開狀態很長時間。 這可能是速度緩慢的原因。

Answer 2

您需要在收到響應后關閉連接。 這似乎引起了問題。