当我尝试使用 selenium xpath 抓取 Web 表格文本值时，它给出了空文本值

Question

I'm trying to get text values from the below URL Annual & quarterly tables. but it gives empty value. what might be the problem here. can anyone give me some help? These are the values I'm trying to extract,

This is the code : in this, I'm trying to get 2018 text. i need all of the text inside the box.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait  # for implicit and explict waits
from selenium.webdriver.chrome.options import Options  # for suppressing the browser
import os

url = f'https://www.morningstar.ca/ca/report/stocks/financials.aspx?t=0P000000GY&lang=en-CA'

codePath = os.getcwd() + r"\chromedriver_win32\chromedriver"
PATH = os.path.join(codePath)
service = Service(PATH)

service.start()
option = webdriver.ChromeOptions()
# option.add_argument('headless')

driver = webdriver.Remote(service.service_url, options=option)

driver.get(url);
time.sleep(2) 

Y1 = driver.find_elements_by_xpath('/html/body/div[3]/div[1]/div/div[1]/div[1]/div[4]/sal-components/section/div/div/div[2]/div/div[2]/div/div/div[2]/div[2]/div/div[2]/div/div[2]/div[2]/div/div[2]/div[2]/div/div/table[2]/tbody/tr[1]/th[2]/span')[0].text
print(Y1)

driver.close()

Answer 1

I am not very sure what you're trying to do but to get table text value just need to do the following:

Assuming that your code snippet works fine and the problem is just with Y1, you can do the following:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait  # for implicit and explict waits
from selenium.webdriver.chrome.options import Options  # for suppressing the browser
import os

url = f'https://www.morningstar.ca/ca/report/stocks/financials.aspx?t=0P000000GY&lang=en-CA'

codePath = os.getcwd() + r"\chromedriver_win32\chromedriver"
PATH = os.path.join(codePath)
service = Service(PATH)

service.start()
option = webdriver.ChromeOptions()
# option.add_argument('headless')

driver = webdriver.Remote(service.service_url, options=option)

driver.get(url);
time.sleep(2) 

Y1 = WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH, "//table[@class='report-table ng-isolate-scope']")))

print(Y1.text)

driver.close()

Is it necessary to use WebDriverWait? The answer is no, i did it with WebDriverWait because you imported it and it was not being used.

The above code prints table text value:

Fiscal 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 TTM 5-Yr Index
103.40
Return on Assets % 27.07 28.54 19.34 18.01 20.45 14.93 13.87 16.07 15.69 17.33 23.21 16.19 7.11
Return on Equity % 41.67 42.84 30.64 33.61 46.25 36.90 36.87 49.36 55.92 73.69 103.40 51.97 22.62
Return on Invested Capital % 41.04 42.01 26.08 26.20 31.32 21.95 19.86 24.41 25.75 30.11 40.07 24.98 12.64

Answer 2

You are using a wrong locator.
Also you have to use expected conditions to wait until the element is loaded. I see it sometimes takes a long time here.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait  # for implicit and explict waits
from selenium.webdriver.chrome.options import Options  # for suppressing the browser
import os

url = f'https://www.morningstar.ca/ca/report/stocks/financials.aspx?t=0P000000GY&lang=en-CA'

codePath = os.getcwd() + r"\chromedriver_win32\chromedriver"
PATH = os.path.join(codePath)
service = Service(PATH)

service.start()
option = webdriver.ChromeOptions()
# option.add_argument('headless')

driver = webdriver.Remote(service.service_url, options=option)

driver.get(url);

Y1 = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "(//div[@class='sal-components-scrollable ng-isolate-scope'])[1]")))

print(Y1.text)

driver.close()