HttpWebRequests blocking UI thread from another thread

Question

I am doing multiple HttpWebRequests on Seaprate threads but my UI Thread is blocked almost all the time after I call Start() and I can't figure out why, here is the most important code:

private void Start()
{
    for (int i = 0; i < 10; i++)
    {
        var todo = UrlStack.UrlPartitions.First();
        UrlStack.UrlPartitions.Remove(todo);
        ThreadPool.QueueUserWorkItem(new WaitCallback(ScanSites), todo);
    }
}
private void ScanSites(object o)
{
    var ToDo = (List<string>)o;
    foreach (string Url in ToDo)
    {
        var state = MakeRequest(Url);
        var website = WebsiteProcessing.ProcessResponse(state);
        if (website != null)
        {
            Websites.Add(website);
            WebsitesCount++;
            this.UIThread(() => lblCrawledWebsites.Text = WebsitesCount.ToString());
        }
    }
}
private static RequestState MakeRequest(string url)
{
    HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
    ...
    RequestState state = new RequestState();
    HttpWebResponse response = null;
    Stream responseStream = null;
    StreamReader sr = null;

    try
    {
        response = (HttpWebResponse)request.GetResponse();
        responseStream = response.GetResponseStream();
        sr = new StreamReader(responseStream);
        state.Request = (HttpWebRequest)request;
        state.Response = (HttpWebResponse)response;
        string strContent = sr.ReadToEnd();
        state.ResponseContent = new StringBuilder(strContent);
    }
    catch
    {
        return null;
    }

    finally
    {
        if (responseStream != null)
            responseStream.Close();
        if (sr != null)
            sr.Close();
        if (response != null)
            response.Close();
    }

    return state;
}

Answer 1

I found the fault which suprised me very much. It was a custom class with 20-30 regexes used to strip leftover HTML after pulling text with HtmlAgilityPack, what I still don't understand is how can background thread block UI, CPU spikes were never above 50%.

Answer 2

Probably because of this call?

this.UIThread(() => lblCrawledWebsites.Text = WebsitesCount.ToString()); 

You didn't provide the definition of UIThread, but looks like it's marshalling calls to the UI thread. In this case, if you have many URLs in the ToDo list, it may choke the UI thread.