[英]C# Parallel WebClient - The operation has timed out
I am a newbie and wanted to know why I am experiencing error on webclient downloadstring()
inside the parallel
. 我是一个新手,想知道为什么我在遇到错误
webclient downloadstring()
内parallel
。 I am out of nowhere of knowledge whether it is because of the my slow connection. 我不知是否是因为我的连接缓慢。 Here is my code:
这是我的代码:
for (int i = 2; i <= 5; i++)
{
string ebayLink = "http://www.ebay.de/sch/Studium-Wissen-/1105/i.html?LH_Auction=1&_sop=1&_nkw=&_pgn=" + i;
//string ebayLink = "http://www.ebay.de/sch/Schule-Ausbildung-/40434/i.html?LH_Auction=1&_sop=1&_nkw=&_pgn=" + i;
ebayLink = "http://www.ebay.de/sch/i.html?LH_Auction=1&_sacat=0&_from=R40&_nkw=B%C3%BCcher&_sop=1&_pgn=" + i;
HtmlWeb hw = new HtmlWeb();
HtmlAgilityPack.HtmlDocument doc = hw.Load(ebayLink);
List<string> eanList = new List<string>();
List<string> links = new List<string>();
foreach (var link in doc.DocumentNode.SelectNodes("//a[@href]"))
{
string url = link.GetAttributeValue("href", "");
if (url.Contains(".de/itm") && !links.Contains(url) && !url.Contains("pt=Zeitschriften") && !url.Contains("pt=Belletristik"))
{
links.Add(url);
}
}
Parallel.ForEach(links, link =>
{
WebClient wc = new WebClient();
string html = wc.DownloadString(link);
EbayItem ebayItem = new EbayItem(html);
string ean = ebayItem.ean;
string amazonUsedPrice = string.Empty;
amazonUsedPrice = getAmazonUsedPrice(ean);
Product product = new Product();
product.EbayUrl = link;
product.Ean = ean;
product.AmazonPriceString = amazonUsedPrice;
product.ebayItem = ebayItem;
productList.Add(product);
}
);}
The error occurs in the string html = wc.DownloadString(link);
错误发生在
string html = wc.DownloadString(link);
. 。 I see on the output that it stops when it reaches at least 20 links.
我在输出中看到,当到达至少20个链接时,它停止。
Your connections are waiting for previous connections to close, thus the timeout. 您的连接正在等待先前的连接关闭,因此超时。 The default limit for concurrent connections to the same host is 2. Try increasing that limit before entering your
Parallel
call: 到同一主机的并发连接的默认限制为2。在进入
Parallel
调用之前,请尝试增加该限制:
System.Net.ServicePointManager.DefaultConnectionLimit = int.MaxValue;
Read more about the DefaultConnectionLimit
here . 在此处阅读有关
DefaultConnectionLimit
更多信息。
Property Value
适当的价值
Type: System.Int32
类型:System.Int32
The maximum number of concurrent connections allowed by a ServicePoint object.
ServicePoint对象允许的最大并发连接数。 The default value is 2.
预设值为2。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.