Trying to get website calendar data using Selenium in python2.7

Question

I'm trying to get all the calendar data split by unavailable and available for this month and next month. I've had no luck in getting the split. Below are the codes I've used. I'm either able to get all the dates without the attribute to identify whether those dates are available or unavailable, or when I try to iterate I get an error "selenium.common.exceptions.StaleElementReferenceException: Message: Element is no longer attached to the DOM"

from selenium import webdriver
url = 'https://www.airbnb.com/rooms/4660676'
driver = webdriver.Firefox()
driver.get(url)
element = WebDriverWait(driver, 10).until(
    EC.element_to_be_clickable((By.XPATH,'/html/body/main/div[3]/div[4]/div/div/div[2]/div[4]/div/div[1]/div/form/div/div/div[2]/button'))
)
element.click()
#This one gets me all the dates 
#listitems=driver.find_elements_by_xpath('/html/body/div[35]/table/tbody/tr[5]')

#If I try something like the following to then be able to get get_attribute then I get the error mentioned above
#available=driver.find_elements_by_xpath('//tr')
#for i in available: print i.text

What would be the way to get the dates. Thanks.

Answer 1

Two things to understand to be able to fix it:

• the datepicker appears right on the button click, but it takes time to load - wait for it
• to determine which of the days is available and not, use the presence of the ui-datepicker-unselectable class

Implementation:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


url = 'https://www.airbnb.com/rooms/4660676'
driver = webdriver.Firefox()
driver.get(url)

wait = WebDriverWait(driver, 10)

# click "Instant Book"
book = wait.until(
    EC.element_to_be_clickable((By.XPATH,'//button[span = "Instant Book"]'))
)
book.click()

# wait for datepicker to load
wait.until(
    EC.visibility_of_element_located((By.CSS_SELECTOR, '.ui-datepicker:not(.loading)'))
)

days = driver.find_elements_by_css_selector(".ui-datepicker table.ui-datepicker-calendar tr td")
for cell in days:
    day = cell.text.strip()
    if not day:
        continue

    if "ui-datepicker-unselectable" in cell.get_attribute("class"):
        status = "Unavailable"
    else:
        status = "Available"

    print(day, status)

Prints:

1 Unavailable
2 Unavailable
3 Unavailable
4 Unavailable
5 Unavailable
6 Unavailable
7 Unavailable
8 Unavailable
9 Unavailable
10 Unavailable
11 Unavailable
12 Unavailable
13 Unavailable
14 Unavailable
15 Unavailable
16 Unavailable
17 Unavailable
18 Unavailable
19 Unavailable
20 Unavailable
21 Unavailable
22 Unavailable
23 Unavailable
24 Unavailable
25 Unavailable
26 Unavailable
27 Unavailable
28 Available
29 Available
30 Unavailable
31 Unavailable