[英]The operation has timed out with WebClient.DownloadFile and correct url's
I am batch uploading products to a database. 我是批量上传产品到数据库。
I am download the image urls to the site to be used for the products. 我将图像网址下载到用于产品的网站。
The code I written works fine for the first 25 iterations (always that number for some reason), but then throws me a System.Net.WebException "The operation has timed out". 我编写的代码在前25次迭代中工作正常(由于某种原因总是那个数字),但随后抛出一个System.Net.WebException“操作已经超时”。
if (!File.Exists(localFilename))
{
using (WebClient Client = new WebClient())
{
Client.DownloadFile(remoteFilename, localFilename);
}
}
I checked the remote url it was requesting and it is a valid image url that returns an image. 我检查了它正在请求的远程URL,它是一个返回图像的有效图像URL。
Also, when I step through it with the debugger, I don't get the timeout error. 此外,当我使用调试器单步执行它时,我没有收到超时错误。
HELP! 救命! ;)
;)
If I were in your shoes, here's a few possibilities I'd investigate: 如果我在你的鞋子里,我会调查一些可能性:
My money is on the latter one, since the fact that it always breaks at a nice round number of requests, and that stepping in the debugger (aka slower!) doesn't trigger the problem. 我的钱是在后一个,因为它总是在一个很好的轮数请求中断,并且踩到调试器(又名更慢!)不会触发问题。
To test all this, I'd start with the easy thing: stick in a delay (Thread.Sleep) before each HTTP call, and see if the problem goes away. 为了测试所有这些,我从简单的事情开始:在每次HTTP调用之前坚持延迟(Thread.Sleep),并查看问题是否消失。 If it does, reduce the delay until the problem comes back.
如果是,请减少延迟,直到问题再次出现。 If it doesn't, increase the delay up to a large number (eg 10 seconds) until the problem goes away.
如果没有,请将延迟增加到大量(例如10秒),直到问题消失。 If it doesn't go away with a 10 second delay, that's truly a mystery and I'd need more info to diagnose.
如果它没有消失10秒延迟,这真的是一个谜,我需要更多的信息来诊断。
If it does go away with a delay, then you need to figure out why-- and whether the limit is permanent (eg server's firewall which you can't change) or something you can change. 如果它确实延迟了,那么你需要找出原因 - 以及限制是否是永久性的(例如服务器的防火墙,你无法改变)或者你可以改变的东西。 To get more info, you'll want to time the requests (eg check DateTime.Now before and after each call) to see if you see a pattern.
要获得更多信息,您需要为请求计时(例如,在每次调用之前和之后检查DateTime.Now)以查看是否看到模式。 If the timings are all consistent and suddenly get huge, that suggests a network/firewall/proxy throttling.
如果时间一致并且突然变得庞大,那表明网络/防火墙/代理限制。 If the timings gradually increase, that suggests a server you're gradually overloading and lengthening its request queue.
如果时间逐渐增加,那表明服务器正在逐渐超载并延长其请求队列。
In addition to timing the requests, I'd set the timeout of your webclient calls to be longer, so you can figure out if the timeout is infinite or just a bit longer than the default. 除了对请求进行计时之外,我还要将webclient调用的超时设置为更长,这样您就可以确定超时是无限的还是仅比默认值更长。 To do this, you'll need an alternative to the WebClient class, since it doesn't support a timeout.
为此,您需要WebClient类的替代方法,因为它不支持超时。 This thread on MSDN Forums has a reasonable alternative code sample.
MSDN论坛上的这个主题有一个合理的替代代码示例。
An alternative to adding timing in your code is to use Fiddler: 在代码中添加时序的另一种方法是使用Fiddler:
it seems that WebClient
is not closing the Response
object it uses when done which will cause, in your case, many responses to be opened at the same time and with a limit of 25 connections on the remote server, you got the 'Timeout exception'. 似乎
WebClient
没有关闭它在完成时使用的Response
对象,这会导致在你的情况下同时打开许多响应并且远程服务器上有25个连接的限制,你得到了'Timeout exception' 。 When you debug, early opened reponses get closed due to their inner timeout, etc... (I inpected WebClient
that with Reflector, I can't find an instruction for closing the response). 当你调试时,早期打开的响应由于它们的内部超时等而被关闭......(我认为使用Reflector的
WebClient
,我找不到关闭响应的指令)。
I propse that you use HttpWebRequest & HttpWebResponse
so that you can clean objects after each download: 我建议你使用
HttpWebRequest & HttpWebResponse
这样你就可以在每次下载后清理对象:
HttpWebRequest request;
HttpWebResponse response = null;
try
{
FileStream fs;
Stream s;
byte[] read;
int count;
read = new byte[256];
request = (HttpWebRequest)WebRequest.Create(remoteFilename);
request.Timeout = 30000;
request.AllowWriteStreamBuffering = false;
response = (HttpWebResponse)request.GetResponse();
s = response.GetResponseStream();
fs = new FileStream(localFilename, FileMode.Create);
while((count = s.Read(read, 0, read.Length))> 0)
{
fs.Write(read, 0, count);
count = s.Read(read, 0, read.Length);
}
fs.Close();
s.Close();
}
catch (System.Net.WebException)
{
//....
}finally
{
//Close Response
if (response != null)
response.Close();
}
Here's a slightly simplified version of manji's answer: 这是manji答案的略微简化版本:
private static void DownloadFile(Uri remoteUri, string localPath)
{
var request = (HttpWebRequest)WebRequest.Create(remoteUri);
request.Timeout = 30000;
request.AllowWriteStreamBuffering = false;
using (var response = (HttpWebResponse)request.GetResponse())
using (var s = response.GetResponseStream())
using (var fs = new FileStream(localPath, FileMode.Create))
{
byte[] buffer = new byte[4096];
int bytesRead;
while ((bytesRead = s.Read(buffer, 0, buffer.Length)) > 0)
{
fs.Write(buffer, 0, bytesRead);
bytesRead = s.Read(buffer, 0, buffer.Length);
}
}
}
I have the same problem and I solve it adding this lines to the configuration file app.config: 我有同样的问题,我解决了将这些行添加到配置文件app.config:
<system.net>
<connectionManagement>
<add address="*" maxconnection="100" />
</connectionManagement>
</system.net>
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.