尝试使用WebClient.DownloadFile（）或WebRequest下载文件时无法实现超时

Question

I'm trying to implement some functionality that downloads a file from a URL. However, if the file is taking longer than 30 seconds, I'd like to cancel the download, or have it time out.

I've tried overriding the WebClient class to implement a timeout, but no matter what value I set the timeout to, it never times out! Here is the code I've tried, found in another stackoverflow answer :

using System;
using System.Net;

public class WebDownload : WebClient
{
    /// <summary>
    /// Time in milliseconds
    /// </summary>
    public int Timeout { get; set; }

    public WebDownload() : this(60000) { }

    public WebDownload(int timeout)
    {
        this.Timeout = timeout;
    }

    protected override WebRequest GetWebRequest(Uri address)
    {
        var request = base.GetWebRequest(address);
        if (request != null)
        {
            request.Timeout = this.Timeout;
        }
        return request;
    }
}

Then, called using:

 WebDownload webClient = new WebDownload(20000);
 try 
 {               
      webClient.DownloadFile(url, tmpFile);
 }
 catch {
     //throw error
 }

I've also tried using the WebRequest method to download the file, and using the Timeout and ReadWriteTimeout properties, but no dice. This has to be a pretty common use case. Any help is appreciated. Thanks!

Answer 1

How about creating an extension method?

WebClient wc = new WebClient();
wc.DownloadFileWithTimeout(url, filename, 20000);

public static class SOExtensions
{
    public static void DownloadFileWithTimeout(this WebClient wc, string url, string file, int timeout)
    {
        var tcs = new TaskCompletionSource<bool>();

        var bgTask = Task.Factory.StartNew(() =>
        {
            wc.DownloadFileTaskAsync(url, file).Wait();
            tcs.TrySetResult(true);
        });


        if (!bgTask.Wait(timeout))
        {
            wc.CancelAsync();
            throw new TimeoutException("Timed out while downloading \"" + url + "\"");
        }
    }
}

Answer 2

The timeout you implemented concerns getting Response, but not ResponseStream which contains all the data and takes usually more time to achieve. For example getting response usually takes below 1 second, but downloading content of a web page could take few seconds.

As for downloading files, you can use HttpWebRequest to get HttpWebResponse and from it use method GetResponseStream() to get the stream to the data.

This will be helpful: Encoding trouble with HttpWebResponse

Especially the part where byte[] buffer is used to get parts of data instead of StreamReader ReadToEnd() method. While downloading parts of data to buffer you may check current DateTime against the timeout DateTime and thus allow cancelling the download after it.

Edit: a useful piece of code

private byte[] DownloadFile( string uri, int requestTimeout, int downloadTimeout, out bool isTimeout, out int bytesDownloaded )
{
    HttpWebRequest request = WebRequest.Create( uri ) as HttpWebRequest;
    request.Timeout = requestTimeout;
    HttpWebResponse response = null;
    Stream responseStream = null;
    MemoryStream downloadedData = null;

    byte[] result = null;
    bytesDownloaded = 0;

    isTimeout = false;
    try
    {
        // Get response
        response = request.GetResponse() as HttpWebResponse;
        byte[] buffer = new byte[ 16384 ];

        // Create buffer for downloaded data
        downloadedData = new MemoryStream();

        // Set the timeout
        DateTime timeout = DateTime.Now.Add( new TimeSpan( 0, 0, 0, 0, downloadTimeout ) );

        // Read parts of the stream
        responseStream = response.GetResponseStream();
        int bytesRead = 0;
        DateTime now = DateTime.Now;
        while ( (bytesRead = responseStream.Read( buffer, 0, buffer.Length )) > 0 && DateTime.Now < timeout )
        {
            downloadedData.Write( buffer, 0, bytesRead );
            now = DateTime.Now;
            bytesDownloaded += bytesRead;
        }

        // Notify if timeout occured (could've been written better)
        if ( DateTime.Now >= timeout )
        {
            isTimeout = true;
        }
    }
    catch ( WebException ex )
    {
        // Grab timeout exception
        if ( ex.Status == WebExceptionStatus.Timeout )
        {
            isTimeout = true;
        }
    }
    finally
    {
        // Close the stream
        if ( responseStream != null )
        {
            responseStream.Close();
        }
    }

    if ( downloadedData != null )
    {
        result = downloadedData.GetBuffer();
        downloadedData.Close();
    }

    return result;
}

Usage

private void button1_Click( object sender, EventArgs e )
{
    bool isTimeout;
    int bytesDownloaded;
    byte[] data = DownloadFile( something, 1000,500, out isTimeout, out bytesDownloaded );

    MessageBox.Show( "Downloaded " + bytesDownloaded.ToString() + " bytes, Timeout = " + isTimeout.ToString() );
}

This code is still vulnerable for other exceptions you may encounter, bear that in mind.