How to find hidden class with Selenium

Question

I am currently working on a demo Selenium project with Python. I have been able to navigate to a page but when trying to collect text within a "div class" selenium fails to find the HTML: Code to be collected

I have made use of the wait functionality but the code still does not find the Html element.

Any suggestions on how to resolve this issue would be appreciated, please see my code below: Image of my selenium

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
import time
import json


# establish Json dict
global data
data = {}

global date
date = '''&checkin=2021-02-22&checkout=2021-02-28&adults=1&source'''

def find_info(place):
    data[place] = []
    driver = webdriver.Chrome('chromedriver.exe')
    driver.get("https://www.airbnb.co.uk/")
    time.sleep(2)

    #first_page_search_bar
    search_bar = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "_1xq16jy")))
    time.sleep(2)
    search_bar.clear()
    time.sleep(2)
    search_bar.send_keys(f"{place}")
    time.sleep(2)
    enter_button = driver.find_element_by_class_name("_1mzhry13")
    enter_button.click()

    #load page
    WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.CLASS_NAME, "_ljad0a")))
    page = driver.current_url
    new_url = page.replace("&source", date)
    # driver = webdriver.Chrome('chromedriver.exe')
    driver.get(new_url)
    time.sleep(3)
    click_button = driver.find_element_by_xpath('//*[@id="menuItemButton-price_range"]/button')
    click_button.click()
    time.sleep(5)
    price = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, '/html/body/div[16]/section/div/div/div[2]/div/section/div[2]/div/div/div[1]')))
    print(price)

find_info("London, United Kingdom")

Answer 1

I've fixed the xpath at the end of your script:

    price = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.XPATH, '(//div[@role="menu"]//div[@dir="ltr"])[1]/preceding-sibling::div')))
    print(price.text)

Explanation: Under the <div role="menu"... there are 3 <div dir="ltr"> elements and the first one happens to be just after the div you are looking for. So we find that one and select the preceding sibling.

Another recommendation: if you replace EC.presence_of_element_located to EC.element_to_be_clickable when you are looking for the input fields at the start you can get rid of a few time.sleep statements.