从源代码中没有的网页中提取数据

Question

I'd like to write a macro in Excel that pulls data from the webpage below:

http://www.richmond.com/data-center/salaries-virginia-state-employees-2013/?appSession=673718284851033&RecordID=101177&PageID=3&PrevPageID=2&cpipage=1&CPIsortType=&CPIorderBy=&cbCurrentRecordPosition=1

The problem I'm having is that the employee information data isn't in the page source so when I use the code below (where NextPage is set to the above URL) the responseText doesn't include the data I'm looking for.

With CreateObject("msxml2.xmlhttp")
    .Open "GET", NextPage, False
    .Send
    htm.body.innerHtml = .responseText
End With

I could very well be wrong but I believe the data is contained within the page's DOM. Can someone help me understand how I can download the contents of this page as displayed (ie after the javascript modifications have been applied) using VBScript?

Answer 1

Using the InternetExplorer.Application COM object should give you access to the actual DOM tree:

url = "http://www.richmond.com/..."

Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True

ie.Navigate url

Do
  WScript.Sleep 100
Until ie.ReadyState = 4

Set elem = ie.Document.getElementById("...")

If that doesn't work, you may have to resort to something like PhantomJS .