selenium Unable to locate element by xpath

Question

i'm trying to scrape web site to csv file, and there is some text elements that i just can't locate.

i keep getting Message: no such element: Unable to locate element:

i'm trying to get the elements with xpath, and i waiting to the site to load before i start to look for them.

my line of code that working for other elements in the site (like the H1 title) is: driver.find_element_by_xpath("//*[contains(@class,'descriptionContainer')]/p[1]").text

and i tried couple of xpath to make sure:

//*[contains(@class,'VenueHeroBanner__description')]/p
//*div[contains(@class,'VenueHeroBanner__description')]/p
//*[contains(@class,'descriptionContainer')]/p[1]
//*[@id='venueHeroBanner']/div[2]/div[1]/p

** and all of them working in the chrome extension called:"xpath helper", but not in my script

note that i opening the the link in a new tab and then trying to get the element if its matters

driver.execute_script(f'''window.open("{rest_links[0]}","_blank");''')

Answer 1

The website seems to need some time to load the data.

You should add expecting conditions to your script.

Imports:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Then use:

WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "your_XPath"))).text
...

Side note: you should fix your last XPath (2 elements are returned).

WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "(//a[contains(@class,'venueInfoLink')])[1]"))).get_attribute("href")

Answer 2

i didnt understand way, but the xpath work if you give him the parent xpath of the child that you want to get

<div class="VenueHeroBanner__descriptionContainer___3Q-jT">
    <p class="VenueHeroBanner__description___1wQwD xh-highlight">Bar & Restaurant</p>

not working-

driver.find_element_by_xpath("//*[contains(@class,'descriptionContainer')]/p[1]").text

that one work:

driver.find_element_by_xpath("//*[contains(@class,'descriptionContainer')]").text

[output] "Bar & Restaurant"