Using selenium VBA bringing the cell values from a dynamic web page table to excel cells

Question

I am new to selenium VBA and post googling have crated the below code in order to bring the each cell value from a dynamic webpage using the selenium vba. I am getting error in receiving the web elements in a web_tr and web_td web elements

Have trying with this line:

ActiveSheet.Cells(i, 1).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[1]").Text

I am getting a header line item first value only though it is under the loop.

Web_tr and web_td are web elements and receiving the web elements to these variables are the issue here. Kindly assist.

---

With ThisWorkbook
        .Sheets("UpfrontOrder#").Activate

       Set web_table = selenium.findElementsByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody")
       Set web_tr = web_table.findElementsByTagName("tr")   '***** the error pop-up gets in here****
       row_count = selenium.findElementsByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr").Count

       For i = 1 To row_count
            Set web_td = web_tr.findElementsByTagName("td")
            ActiveSheet.Cells(i, 1).Value = web_td(1).getText

            'ActiveSheet.Cells(i, 1).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[1]").Text
            ActiveSheet.Cells(i, 2).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[2]").Text
            ActiveSheet.Cells(i, 3).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[3]").Text
            ActiveSheet.Cells(i, 4).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[4]").Text
            ActiveSheet.Cells(i, 5).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[5]").Text
            ActiveSheet.Cells(i, 6).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[6]").Text
            ActiveSheet.Cells(i, 7).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']/td[7]").Text

            'ActiveSheet.Cells(i, 1).Value = web_td.findElementByXPath("td[1]").Text
            'ActiveSheet.Cells(i, 2).Value = web_td(1).Text

        Next i
End With

Answer 1

I'm not really familiar with the VBA version of Selenium but from what I can tell, I see one issue that I think is causing the error you are getting. When you use .findElements* (plural), it will return a collection of elements instead of just a single element. Your first line (below) is using .findElements() but in the second line (below) is not specifying which element of the web_table collection to reference.

Set web_table = selenium.findElementsByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody")
Set web_tr = web_table.findElementsByTagName("tr")   '***** the error pop-up gets in here****

You can try to do one of two fixes...

1. Change .findElements() to .findElement() so that you only grab the first (and probably only?) table. If you only want the first one, this is the right way to fix this.

2. Add an index into the collection of elements from the first line, eg change web_table. to web_table(1). If you want a TABLE tag other than the first one, then add the correct reference.

    Set web_table = selenium.findElementsByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody") Set web_tr = web_table(1).findElementsByTagName("tr") add this _____________^^^ 

---

Additional Note: I had another suggestion to simplify some of your code. In your For i loop you have hardcoded indexes 2-7 where I think you can easily use a loop. I updated the code below. This accomplishes two things...

1. It eliminates the repeated scraping of the page. In your original code, you are scraping one element at a time. In the code below, I grab a collection of TD s and then iterate through them. That way I'm only scraping the page once to get all the elements I care about and then processing that collection. It's more efficient.
2. Simplifies the code.

For i = 1 To row_count
    Set web_td = web_tr.findElementsByTagName("td")
    ActiveSheet.Cells(i, 1).Value = web_td(1).getText
    Dim tds As New List
    Set tds = ActiveSheet.Cells(i, 2).Value = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr['i']")
    For j = 2 To 7
        ActiveSheet.Cells(i, j).Value = tds(j).Text
    Next j
Next i

Answer 2

Thank you both of you.

The below code works well in finding the each cell values.

With ThisWorkbook
        .Sheets("UpfrontOrder#").Activate

       Set web_table = selenium.findElementByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody")
       Set web_tr = web_table.findElementsByTagName("tr")

       row_count = selenium.findElementsByXPath("//table[@id='ctl00_cphContentBody_itemsTable']/tbody/tr").Count

       For i = 0 To (row_count - 2)

            Set web_td = web_tr(i).findElementsByTagName("td")
            For j = 0 To 6
                   ActiveSheet.Cells((i + 1), (j + 1)).Value = web_td(j).Text
            Next j
        Next i
End With