VBA code - Stuck at the 4th loop

Question

I have a code that is supposed to open a website, select a location, copy the HTML table to the Excel sheet and repeat on another location. However when I tried running the 'For' loop, I got error at the 4th iteration. The message said "Object variable or With block variable not set". The debug tool pointed at line 44

Sub ParseTable()
    Dim IE As InternetExplorer
    Dim htmldoc As MSHTML.HTMLDocument 'Document object
    Dim eleColtr As MSHTML.IHTMLElementCollection 'Element collection for tr tags
    Dim eleColtd As MSHTML.IHTMLElementCollection 'Element collection for td tags
    Dim eleRow As MSHTML.IHTMLElement 'Row elements
    Dim eleCol As MSHTML.IHTMLElement 'Column elements
    Dim ieURL As String 'URL
    Dim x As Integer
    Dim y As String

y = "A1"

For x = 1 To 4
    If x <> 2 Then 'Skip iteration 2
        Set IE = New InternetExplorer
        IE.Visible = True
        Application.ScreenUpdating = False

        ieURL = "***"
        IE.navigate ieURL
 'Wait
        Do While IE.Busy Or IE.readyState <> 4
            DoEvents
        Loop

        Set htmldoc = IE.document 'Document webpage

        Do While IE.Busy Or IE.readyState <> 4
            DoEvents
        Loop

    IE.document.getElementById("ddlLevel1").selectedIndex = x
    IE.document.getElementById("ddlLevel1").FireEvent ("onchange")
    Do While IE.Busy Or IE.readyState <> 4
        DoEvents
    Loop
    Set eleColtr = IE.document.getElementsByTagName("tr") 'Find all tr tags

    'This section populates Excel
            i = 0 'start with first value in tr collection
            For Each eleRow In eleColtr 'for each element in the tr collection
                Set eleColtd = IE.document.getElementsByTagName("tr")(i).getElementsByTagName("td") 'get all the td elements in that specific tr
                j = 0 'start with the first value in the td collection
                For Each eleCol In eleColtd 'for each element in the td collection
                    Sheets("Sheet1").Range(y).Offset(i, j).Value = eleCol.innerText 'paste the inner text of the td element, and offset at the same time
                    j = j + 1 'move to next element in td collection
                Next eleCol 'rinse and repeat
                i = i + 1 'move to next element in td collection
            Next eleRow 'rinse and repeat
            Sheets("Sheet1").Range(y).Offset(i, 0).Select
            y = "A" & ActiveCell.Row
            IE.Quit
      End If
 Next x
 Application.ScreenUpdating = True
 End Sub

Not sure what could be the possible cause. I did get the tables from first 4 locations (minus location 2) on my Excel sheet. Pardon for my long, inefficient code (I am not a programmer myself). The webpage that I use requires login and has confidential data, but I will try to provide input as much as possible. Thanks in advance for any help!

Answer 1

I see this happen in a lot of examples that try to wait for IE.Busy before processing further. The problem is that the document model hasn't quite formed yet so you get an error trying to access the element in the object model.

The safest way is to actually loop until the object exists - ie. if you know for a fact that it will appear in each document. If you want to play safe, you can add a limit on the number of checks you do before you stop with an error message.

Add this to top of your module

Private Declare Sub Sleep Lib "kernel32" (ByVal dwMilliseconds As Long)

Then replace this line (if it's the problem line)

Set eleColtr = IE.document.getElementsByTagName("tr")  

With this slight Delay and DoEvents - then a loop to check for your tr elements

DoEvents
Sleep 1000 ' delay once second
Do 
   DoEvents
   Sleep 500 ' delay half a second
Loop Until not IsNull(IE.document.getElementsByTagName("tr"))

If it goes into an endless loop, not finding your tr elements, you can put a counter on number of tries

I'd also add a delay at the start of each loop

After For Each eleRow In eleColtr add Sleep 500