Scraper不会停止点击下一页按钮

Question

I've written a script in python in combination with selenium to get some names and corresponding addresses displayed upon a search and the search keyword is "Saskatoon". However, the data, in this case, traverse multiple pages. My script almost does everything except for one thing.

1. It still runs even though there are no more pages to traverse. The last page also holds ">" sign for next page option and is not grayed out.

Here is the link: Page_link

Search_keyword: Saskatoon (in the city/town field).

Here is what I've written:

from selenium import webdriver; import time
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("above_link")
time.sleep(3)

search_input = driver.find_element_by_id("cityField")
search_input.clear()
search_input.send_keys("Saskatoon")
search_input.send_keys(Keys.ENTER)

while True:
    try:
        wait.until(EC.visibility_of_element_located((By.LINK_TEXT, "›"))).click()
        time.sleep(2)
    except:
        break
driver.quit()

BTW, I've just taken out the name and address part form this script which I suppose is not relevant here. Thanks.

Answer 1

您可以使用>按钮的class属性，如在最后一页上"ng-scope disabled" "ng-scope" - 在其余页面上 - "ng-scope" ：

wait.until(EC.visibility_of_element_located((By.XPATH, "//li[@class='ng-scope']/a[.='›']"))).click()