How to obtain required content from “find_element_by_xpath”

Question

I'm trying to use selenium & python to auto get part of the html content. I use find xpath like below ways to sort out the price from specified flight number, but always get failed result " unable to locate " Anyone could shed some lights on it ?

element_price = driver.find_element_by_xpath("//div[@id='flight_MU5401']")
element_price.find_element_by_xpath(".//span[@class='base_price02']")

It's the html

Answer 1

If I were to guess, I would say that you are probably getting that error because the element isn't loaded when your code runs. It's probably a slower loading page. You should try adding a wait and see if that helps.

You can also simplify your locator by using a CSS selector and only scraping the page once.

price = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#flight_MU5401 span.base_price02")).text