I am new to web scraping and am trying to scrape data from this real estate website to get only the places that have recently been rented. To do this I need to click "Leased Listing" from this dropdown menu. Picture of what I need to click
The issue I am having is this is not a button class so using the selenium.click() function is giving me an error. There are also multiple objects with the same class name as the "Leased Listing" section.
Here is my code:
for page in range(0, total_pages + 1):
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('chromedriver',chrome_options=chrome_options)
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0 ; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36'}
url = 'https://www.zoocasa.com/toronto-on-real-estate-for-rent?page=' + str(page)
driver.get(url)
elements = driver.find_elements(By.CLASS_NAME, "style_component__DR_Bs")
elements[6].click() #Leased listing is the 7th appearance of this class name
And here is the site's html (whatever is clicked has the "style_active__eGbvT"):
<div class="style_component__DR_Bs">
::before
Active Listing
::after
</div>
<div class="style_component__DR_Bs style_active__eGbvT">
::before
Leased Listing
::after
</div>
<div class="style_component__DR_Bs">
::before
Expired Listing
::after
</div>
If anyone has any suggestions I would really appreciate it, Thanks.
sorry cannot comment due to my reputation <50
I think selenium allows to click div
However, you need to identify the correct div
and id
before proceeding.
The easiest way, can you driver.find_elements_by_xpath
, and it will return a list then you need to check text
for each item
Code:
list_subcontent = driver.find_elements_by_xpath(".//div[@class='style_component__lT4sh style_theme-dark__rZF3s']//div[@class='style_component__DR_Bs ']")
for item in list_subcontent:
if item.text == 'Leased Listing':
item.click()
help to upvote if this help
There are 2 problems here:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver.get(url)
wait = WebDriverWait(driver, 20)
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[contains(@class,'style_theme-dark')]//div[contains(text(),'Active Listings')]"))).click()
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[contains(@class,'style_theme-dark')]//div[contains(text(),'Leased Listing')]"))).click()
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.