Error when getting website table data using python selenium - Multiple tables and Unable to locate element

Question

I am trying to get info from brazilian stock market (BMF BOVESPA). The website has several tables, but my code is not being able to get them.

The code below aims to get all data from table "Ações em Circulação no Mercado" -> one of the last tables from webpage.

I have tried the ones below, but none worked for me:

content = browser.find_element_by_css_selector('//div[@id="div1"]')

and

table = browser.find_element_by_xpath(('//*[@id="div1"]/div/div/div 1 /table/tbody'))

Thanks in advance for taking my question.

from selenium import webdriver
from time import sleep

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-
Listadas/ResumoEmpresaPrincipal.aspx?codigoCvm=19348&idioma=pt-br"
browser = webdriver.Chrome()
browser.get(url)
sleep(5) #wait website to reload
content = browser.find_element_by_css_selector('//div[@id="div1"]')

HTML can be found at attached picture

As alternative, the code below reaches the same website

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-Listadas/BuscaEmpresaListada.aspx?idioma=pt-br"
Ticker='ITUB4'
browser = webdriver.Chrome()
browser.get(url)
sleep(2)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_txtNomeEmpresa_txtNomeEmpresa_text"]')).send_keys(Ticker)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_btnBuscar"]')).click();
content = browser.find_element_by_id('div1')

Answer 1

Selenium with Python documentation UnOfficial

Hii there

Selenium provides the following methods to locate elements in a page:

find_element_by_id
find_element_by_name
find_element_by_xpath
find_element_by_link_text
find_element_by_partial_link_text
find_element_by_tag_name
find_element_by_class_name
find_element_by_css_selector

Why your code doesnt work ? because you're not using correct correct code to locate element

you're using xpath inside css selector

content = browser.find_element_by_css_selector('//div[@id="div1"]') #this part is wrong

instead you can do this if you want to select div1

content = browser.find_element_by_id('div1')

here's the correct code

url = "http://bvmf.bmfbovespa.com.br/cias-Listadas/Empresas-

Listadas/BuscaEmpresaListada.aspx?idioma=pt-br"
Ticker='ITUB4'
browser = webdriver.Chrome()
browser.get(url)
sleep(2)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_txtNomeEmpresa_txtNomeEmpresa_text"]')).send_keys(Ticker)
browser.find_element_by_xpath(('//*[@id="ctl00_contentPlaceHolderConteudo_BuscaNomeEmpresa1_btnBuscar"]')).click()

I tested it and it worked :)

Mark it as best answer if i helped you :)