VBA: set HTML as Internet Explorer object ERROR

Question

I'm trying to get the content from a website with VBA, however I keep getting an error.

I already tried several other ways that I looked into similar questions, but nothing seems to work...I also tried with set ie = New InternetExplorer but it didn't work either

Can you help me? My goal is further to find a specific key word and count the number of times it appears.

Thanks, M

Sub website()

Dim ie As Object
Dim ht As HTMLDocument

Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = False

ie.navigate ("https://poetenalinha.pt/2021/01/13/gratinado-de-massa-com-peru-e-legumes/")

Set ht = ie.document --> here is the error!!

Set elems = ht.getElementsByClassName("entry-content")

For Each elem In elems
    Debug.Print (elem.innerText)
    
Next

End Sub

Answer 1

This worked for me - for some reason the ie object gets disconnected from the browser instance. If you re-make the connection (using the "GetIE" function) it works.

Sub website()

    Dim ie As Object, elems, elem
    Dim ht As HTMLDocument
    
    Set ie = CreateObject("InternetExplorer.Application")
    ie.Visible = True
    
    ie.navigate "https://poetenalinha.pt/2021/01/13/gratinado-de-massa-com-peru-e-legumes/"

    Application.Wait Now + TimeSerial(0, 0, 10) ' wait 10 sec
    
    Set ie = GetIE("https://poetenalinha.pt")   ' "reconnect" to IE object
    Set ht = ie.document
    
    Set elems = ht.getElementsByClassName("entry-content")
    
    For Each elem In elems
        Debug.Print (elem.innerText)
    Next

End Sub

'get a reference to an existing IE window, given a partial URL
Function GetIE(sLocation As String) As Object

    Dim objShell As Object, objShellWindows As Object, o As Object
    Dim sURL As String
    Dim retVal As Object

    Set retVal = Nothing
    Set objShell = CreateObject("Shell.Application")
    Set objShellWindows = objShell.Windows

    For Each o In objShellWindows
        sURL = ""
        On Error Resume Next  'because may not have a "document" property
        'Check the URL and if it's the one you want then
        ' assign the window object to the return value and exit the loop
        sURL = o.document.Location
        On Error GoTo 0
        If sURL Like sLocation & "*" Then
            Set retVal = o
            Exit For
        End If
    Next o

    Set GetIE = retVal

End Function

Answer 2

You need to wait until IE fully loaded after navigating to the website. The complete code is like below and works well:

Sub website()

Dim ie As Object
Dim ht As HTMLDocument

Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True

ie.navigate ("https://poetenalinha.pt/2021/01/13/gratinado-de-massa-com-peru-e-legumes/")

'add this part
Do Until ie.readyState = 4
   DoEvents
Loop
    
Set ht = ie.document

Set elems = ht.getElementsByClassName("entry-content")

For Each elem In elems
    Debug.Print (elem.innerText)
    
Next

End Sub

Result: