使方法异步运行的最佳方法是什么？

Question

Hi i'm new in async programming. How can I run my method checkAvaible to run async? I would like to download 2500 pages at once if it's possible, dont wait to complete one download and start another. How can I make it?

private static void searchForLinks()
    {
        string url = "http://www.xxxx.pl/xxxx/?action=xxxx&id=";


        for (int i = 0; i < 2500; i++)
        {
            string tmp = url;
            tmp += Convert.ToString(i);

            checkAvaible(tmp); // this method run async, do not wait while one page is downloading
        }

        Console.WriteLine(listOfUrls.Count());
        Console.ReadLine();
    }

    private static async void checkAvaible(string url)
    {
        using (WebClient client = new WebClient())
        {
            string htmlCode = client.DownloadString(url); 

            if (htmlCode.IndexOf("Brak takiego obiektu w naszej bazie!") == -1)
                listOfUrls.Add(url);
        }
    }

Answer 1

1. You would not want to download 2500 pages at the same time since this will be a problem for both your client and the server. Instead, I have added a concurrent download limitation (of 10 by default). The web pages will be downloaded 10 page at a time. (Or you can change it to 2500 if you are running a super computer :))
2. Generic Lists (I think it is a List of strings in your case) is not thread safe by default therefore you should synchronize access to the Add method. I have also added that.

Here is the updated source code to download pages asynhcronously with a configurable amount of concurrent calls

private static List<string> listOfUrls = new List<string>();

private static void searchForLinks()
{
    string url = "http://www.xxxx.pl/xxxx/?action=xxxx&id=";

    int numberOfConcurrentDownloads = 10;

    for (int i = 0; i < 2500; i += numberOfConcurrentDownloads)
    {
        List<Task> allDownloads = new List<Task>();
        for (int j = i; j < i + numberOfConcurrentDownloads; j++)
        {
            string tmp = url;
            tmp += Convert.ToString(i);
            allDownloads.Add(checkAvaible(tmp));
        }
        Task.WaitAll(allDownloads.ToArray());
    }

    Console.WriteLine(listOfUrls.Count());
    Console.ReadLine();
}

private static async Task checkAvaible(string url)
{
    using (WebClient client = new WebClient())
    {
        string htmlCode = await client.DownloadStringTaskAsync(new Uri(url));

        if (htmlCode.IndexOf("Brak takiego obiektu w naszej bazie!") == -1)
        {
            lock (listOfUrls)
            {
                listOfUrls.Add(url);
            }
        }
    }
}

Answer 2

It's best to convert code to async by working from the inside and proceeding out. Follow best practices along the way, such as avoiding async void , using the Async suffix, and returning results instead of modifying shared variables:

private static async Task<string> checkAvaibleAsync(string url)
{
  using (var client = new HttpClient())
  {
    string htmlCode = await client.GetStringAsync(url); 

    if (htmlCode.IndexOf("Brak takiego obiektu w naszej bazie!") == -1)
      return url;
    else
      return null;
  }
}

You can then start off any number of these concurrently using Task.WhenAll :

private static async Task<string[]> searchForLinksAsync()
{
  string url = "http://www.xxxx.pl/xxxx/?action=xxxx&id=";

  var tasks = Enumerable.Range(0, 2500).Select(i => checkAvailableAsync(url + i));
  var results = await Task.WhenAll(tasks);
  var listOfUrls = results.Where(x => x != null).ToArray();

  Console.WriteLine(listOfUrls.Length);
  Console.ReadLine();
}