Excel VBA: Wait for JavaScript execution in Internet Explorer

Question

I am trying to do some web scraping in Excel VBA. Here is the part of the code that I am having trouble with:

IE.Navigate URL
Do
  DoEvents
Loop While IE.ReadyState <> 4 Or IE.Busy = True
Set doc = IE.document

After running this doc contains html that still has unexecuted JavasScript in it. This is the signature of the script that has not been executed:

<SCRIPT type=text/javascript>
        goosSearchPage.Initialize(...)...;
</SCRIPT>

I can wait for execution by doing Application.Wait(Now + TimeValue(x)) but that really is not satisfactory, as the amount of time the script takes to execute is quite variable depending on input.

Is there a way to either wait for the script to finish evaluating or to just evaluate the script directly in the doc object?

Answer 1

I found code that does wait for a page to complete. per the notes here , it requires the Microsoft Internet Controls as a reference in your code.

Code reproduced here, just in case the link dies:

'Following code goes into a sheet or thisworkbook class object module
Option Explicit

'Requires Microsoft Internet Controls Reference Library
Dim WithEvents ie As InternetExplorer
Sub start_here()
  Set ie = New InternetExplorer
  'Here I wanted to show the progress, so setting ie visible
  ie.Visible = True
  'First URL to go, next actions will be executed in
  'Webbrowser event sub procedure - DocumentComplete
  ie.Navigate "www.google.com"
End Sub
Private Sub ie_DocumentComplete(ByVal pDisp As Object, URL As Variant)
  'pDisp is returned explorer object in this event
  'pDisp.Document is HTMLDocument control that you can use
  'Following is a choice to follow,
  'since there is no do-loop, we have to know where we are by using some reference
  'for example I do check the URL and do the actions according to visited URL

  'In this sample, we use google entry page, set search terms, click on search button
  'and navigate to first found URL
  'First condition; after search is made
  'Second condition; search just begins
  If InStr(1, URL, "www.google.com/search?") > 0 Then
    'Open the first returned page
    ie.Navigate pDisp.Document.getelementsbytagname("ol")(0).Children(0).getelementsbytagname("a")(0).href
  ElseIf InStr(1, URL, "www.google.com") > 0 Then
    pDisp.Document.getelementsbyname("q")(0).Value = "VB WebBrowser DocumentComplete Event"
    pDisp.Document.getelementsbyname("btnG")(0).Click
  End If
End Sub

Answer 2

You actually can evaluate the javascript function with the ie window. But you gotta set up a Callback because the function will be evaluated async.

Answer 3

This post is quite old, but I'll answer this also as a reply to myself, now that I've discovered how to do this.

Simply point to a content you expect to be there after the jQuery script has run, trigger the desired event using JavaScript ran through IE automation, and do a Loop to wait until the desired content appears.

'This will trigger the jQuery event.
Doc.parentWindow.execScript "$('#optionbox').trigger('change')"

'This is the code that will make you wait. It's surprisingly efficient
Do While InStrB(Doc.getElementById("optionbox").innerHTML, "<desired html tag>") = 0
    DoEvents
Loop