登录网站后，Selenium 驱动程序找不到任何元素 [Python]

Question

First of all if a similar topic occurred earlier I'm sorry but I couldn't find any problem like mine.

I would like to create a simple script which enters an e-mail website, log into my account and finds the amount of unread messages.

This is the part with logging in

from selenium import webdriver
from time import sleep

class sMailBot():
    def __init__(self):
        self.driver = webdriver.Chrome()

    def login(self):
        self.driver.get('website.com')

        sleep(2)

        btn_login = self.driver.find_element_by_xpath('//*[@id="username"]')
        btn_login.send_keys('my_username')

        btn_password = self.driver.find_element_by_xpath('//*[@id="password"]')
        btn_password.send_keys('my_password')

        btn_logintoaccount = self.driver.find_element_by_xpath('//*[@id="button"]')
        btn_logintoaccount.click()

        sleep(5)

It works really well. After logging into my mail account comments like driver.title or driver.current_url work.

Now I would like to scrape this part of html code:

<b>some_important_string_which_stores_the_amount_of_unread_mails</b>

I tried to do this using it's path

driver.find_element_by_xpath('//*[@id="MS_act1"]/span)

However it does not work. Moreover I can't find any other elements from this side.

I would like to highlight that I waiting even more than 10 seconds for the page to load.

The error which occurred

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="MS_act1"]/span/b"}
  (Session info: chrome=80.0.3987.87)

As you asked I add some surrounding HTML code

<span style="float: right">
<b>some_important_string_which_stores_the_amount_of_unread_mails</b> 
</span>

Answer 1

Please, don't use sleep , it's not a good choice for selenium.

Instead, use selenium waits:

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )
finally:
    driver.quit()

https://selenium-python.readthedocs.io/waits.html

Answer 2

First of all I will avoid using sleep. You may try using WebDriverWait instead. This will pause the browser until a given condition is satisfied.

eg as follows

WebDriverWait(self.driver, 60).until(EC.presence_of_element_located((By.XPATH, "//button[text()='Login']")))

This will wait for 60 sec maximum for the element (button with text Login) to occur in the page.

Answer 3

After logging into your mail account commands like driver.title and driver.current_url works but they are not part of the DOM Tree .

The relevant HTML would have helped us to construct a canonical answer. However to extract the desired text, you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies :

• Using CSS_SELECTOR and get_attribute("innerHTML") :

   print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "[id^='MS_act'] span>b"))).get_attribute("innerHTML"))

• Using XPATH and text attribute:

   print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[starts-with(@id, 'MS_act')]//span/b"))).text)

• Note : You have to add the following imports :

   from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC