Webbrowser Trying to parse some page which uses ajax to load data

Question

I am trying to parse some images from this site.. I was using htmlagilitypack for the other pages but this page uses ajax to load images

so this is how the webpage works.

1. has a div tag including nothing.

2. right below the div they have a script tag in cdata thing

    <script type="text/javscript> //< ![CDATA[ (function(){ ajax POST request to a 'aaaaaa.js' file with the id parameter and if the request is success, it updates the blank div my changing the innerHTML value. })(); //]]> </script> 

So.. what I tried was...

1. navigate to that page using the webbrowser control. which loads the image just fine but when I try to get the value of 'DocumentText' it only shows the blank div tag...

2. try to get straight data from the ajax POST using webrequest and webresponse.. but.. maybe cause its .js file.. it doesnt work.. I only get http errors.

3. Browse right to the js file with the parameters attached. gives me an error page

4. Browse the page Im trying to parse and then navigate to the .js page. (I guess the browser caches something when i browse the original page.. but i dont know what it is.) I do get the json response! i can use this data.. but since the webbrowser control is using IE. It just asks me if i want to download the responsed js file.

Answer 1

So theres the method of using the DOM I mentioned in my comment.

Another method could be to use FiddlerCore ( http://www.fiddler2.com/fiddler/Core/ )

Or make the ajax call yourself. You will have to make sure you respect cookies, redirects, and all the headers.