Problem using the webbrowser control

Question

I am working on scraper and using webbrowser control. Now I'm navigating pages 1 by 1 in the same control.

but while i do this sometimes the browser start loading other page before the first page has been completely loaded. while sometimes it stop all the execution and no page is url is under progress.

I am using the webBrowser1_DocumentCompleted event to process the document and sometime it is been loaded more than once.

So is there any solution that i can check whether my webbrowser is busy working or has done with browsing.

Answer 1

I've noticed it does document completed multiple times if the page uses frames.
You can get the URL of the loaded page from the WebBrowser1_DocumentCompleted Event Arguments.

If it is the case that there's multiple frames, you could compare URL of WebBrowser1_DocumentCompleted with WebBrowser1.URL

so

if(e.URL == webBrowser1.url) {
  // The main page is done loading
}

Or you can try checking

WebBrowser1.ReadyState == WebBrowserReadyState.Complete

A couple gotchas I've found with the WinForms WebBrowser control are

The DOM you have access to doesn't update when changes are done thru AJAX.

Unless you do some extra work to detect when a new window gets created and put that into a tab, (there's some articles on how to do that, if you need it I can send a link) pages that open new windows will be started in a new process so you lose the ability to scrape those and they loose the session, so if you're logged into a webpage you'll be logged out in the new window.