Parsing HTML in C# that is updating constantly

Question

I have a webpage that is displaying some data using AJAX queries. I would need to parse some of this data in a C# program.

Problem is that when I look at the source code of my webpage, this is not showing up the data, as this is being generated automatically by an AJAX script and modifying the DOM.

If I select everything on the webpage and do "Inspect Element" with Chrome, I have the full HTML code with the data I want to extract that are in various tables.

What I've tried is doing a webBrowser1.Navigate("www.site.com") , and then in my webBrowser1_DocumentCompleted() event, I'm doing this:

var name = webBrowser1.Document.GetElementById("table_1_r_7_c_2");

Problem is that webBrowser1 is not returning the full HTML code, as some code is generated by the AJAX queries.

Does anyone know how I could achieve this behavior in C#?

Answer 1

The DocumentCompleted event is a bit misleading because it will also fire for each AJAX request on the page. You can do something like this to check if it's the actual page that's loaded, or some other variant to look for specific requests.

  private void OnDocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
    {
       if (e.Url.AbsolutePath == webBrowser1.Url.AbsolutePath)
       {
          // page loaded
       }
    }