How to get hyperlinks from internet explorer from VBA

Question

I am trying to get the links for the various sectors (Basic materials, Conglomerates, etc) within the https://biz.yahoo.com/p/s_conameu.html website.

I have the code below, but since this site does not use ids to set what these links will be, I don't know how to actually find the links. Also, the links are actually only partial. I know what I would need to add to make them full (add https://biz.yahoo.com/p/ in front of the partial link in the html file. the first link I see appears on line 238 of the html, but I'm not sure how to actually search and find this as the names of the sectors may change in the future. I'm trying to make it agnostic where it can always search this list and pull all of the links.

Here is a snapshot of the html showing the link (starts on line 236):

nowrap
bgcolor=ffffee><a
href=1conameu.html><font
face=arial
size=-1>Basic Materials</a></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>-0.13</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>293348.2B</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>17.42</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>6.50</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>4.12</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>69.76</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>3.09</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>1.35</font></td><td
align=right
bgcolor=ffffff><font
face=arial
size=-1>6.48</font></td></tr><tr><td
nowrap
bgcolor=ffffee><a
href=2conameu.html><font
face=arial
size=-1>Conglomerates</a></td><td

Here is the code where I grab the website and get the contents.

Public Sub clicklick()
Dim internet As Object
Dim html As HTMLDocument
Dim internetdata As Object
Dim div_result As Object
Dim header_links As Object
Dim link As Object
Dim URL As String
Dim i As Integer

Set internet = CreateObject("InternetExplorer.Application")
internet.Visible = False

URL = "https://biz.yahoo.com/p/s_conameu.html"
internet.Navigate URL

While internet.Busy
    DoEvents
Wend

Application.Wait Now + TimeSerial(0, 0, 5)

Set internetdata = internet.Document
Set div_result = internetdata.getElementById("res")  // This does not work (obviously, but not sure how to really search).  Returns nothing.

Set header_links = div_result.getelementsbytagname("h3")    //This fails because div_result has nothing.
MsgBox html.DocumentElement.innerHTML
MsgBox div_result
SPws.Cells.ClearContents
For Each h In header_links
    Set link = h.ChildNodes.Item(0)
    SPws.Cells(Range("A" & Rows.count).End(xlUp).row + 1, 1) = link.href
Next

MsgBox "done"

End Sub

Thank you in advance for any help. I'm hoping I can do what I want so I can automate some searches.

thank you, Alan

Answer 1

You can do this. It will pull all the links on the page and then you can have it print out the values in cells.

x = 1
For Each link In internet.document.Links
    Cells(x, 1) = link
    'or you can add your prefix URL and link at the same time
    Cells(x, 1) = "https://biz.yahoo.com/p/" & link
    x = x + 1
Next

Answer 2

This may work for you.

Sub Macro1()

    Range("A1").Select
    With ActiveSheet.QueryTables.Add(Connection:= _
        "URL;https://biz.yahoo.com/p/s_conameu.html", Destination:=Range("$A$1"))
        .CommandType = 0
        .Name = "s_conameu_1"
        .FieldNames = True
        .RowNumbers = False
        .FillAdjacentFormulas = False
        .PreserveFormatting = True
        .RefreshOnFileOpen = False
        .BackgroundQuery = True
        .RefreshStyle = xlInsertDeleteCells
        .SavePassword = False
        .SaveData = True
        .AdjustColumnWidth = True
        .RefreshPeriod = 0
        .WebSelectionType = xlSpecifiedTables
        .WebFormatting = xlWebFormattingNone
        .WebTables = "4"
        .WebPreFormattedTextToColumns = True
        .WebConsecutiveDelimitersAsOne = True
        .WebSingleBlockTextImport = False
        .WebDisableDateRecognition = False
        .WebDisableRedirections = False
        .Refresh BackgroundQuery:=False
    End With
End Sub