How do I recover the full html of a page, including what is generated by javascript

Question

How do I recover the full html of a page, including what is generated by javascript. The problem is that I want to access the contents of the select tag, but the page but it is coming empty, this probably being generated dynamically. Please I'm about to give up! I just posted a piece of code because this very large, if I find it necessary to put the whole code.

res = (HttpWebResponse)req.GetResponse();
res.Cookies = req.CookieContainer.GetCookies(req.RequestUri);
cookieContainer.Add(res.Cookies);

sr = new StreamReader(res.GetResponseStream());
getHtml = sr.ReadToEnd();
viewstate = rxViewstate.Match(getHtml).Groups[1].Value;
EventValdidation = rxEventValidation.Match(getHtml).Groups[1].Value;
viewstate = HttpUtility.UrlEncode(viewstate);
EventValdidation = HttpUtility.UrlEncode(EventValdidation);

//Here I should take the contents of the select tag.
getHtml = rxDropDownMenu.Match(getHtml).Groups[2].Value;

Answer 1

You can't just do this with HttpWebRequest , all that does is download the raw HTML and non of the linked JavaScript files.

It also wouldn't run the JavaScript or give you any kind of DOM to inspect.

You'd really need to use WebBrowser or perhaps something like Awesomium .