如何移動到 Python Selenium 的下一頁？

Question

我正在嘗試為特定站點構建代理抓取工具，但無法移至下一頁。

這是我正在使用的代碼。

如果您回答我的問題，請向我解釋一下您使用的是什么，如果可以的話，如果有任何關於此類代碼的好教程，請給我一些：

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time

options = Options()
#options.headless = True     #for headless
#options.add_argument('--disable-gpu') #for headless and os win

driver = webdriver.Chrome(options=options)

driver.get("https://hidemyna.me/en/proxy-list/")
time.sleep(10) #bypass cloudflare


tbody = driver.find_element_by_tag_name("tbody")
cell = tbody.find_elements_by_tag_name("tr")

for column in cell:
    column = column.text.split(" ")
    print (column[0]+":"+ column[1]) #ip and port

nxt = driver.find_element_by_class_name('arrow_right')
nxt.click()

Answer 1

要移動到下一頁，您可以嘗試以下解決方案：

• 代碼塊：

   from selenium import webdriver from selenium.webdriver.chrome.options import Options from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.common.exceptions import TimeoutException, WebDriverException options = Options() options.add_argument("start-maximized") options.add_argument("disable-infobars") options.add_argument("--disable-extensions") driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe') driver.get('https://hidemyna.me/en/proxy-list/') while True: try: driver.execute_script("return arguments[0].scrollIntoView(true);", WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//li[@class='arrow__right']/a")))) driver.find_element_by_xpath("//li[@class='arrow__right']/a").click() print("Navigating to Next Page") except (TimeoutException, WebDriverException) as e: print("Last page reached") break driver.quit()

• 控制台輸出：

   Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page Navigating to Next Page. . . Navigating to Next Page Last page reached

Answer 2

您實際上並沒有點擊錨點<a>標簽。 要導航到下一頁，您需要click <a>鏈接。

您可以像下面這樣使用find_element_by_xpath 。

driver.find_element_by_xpath('//*[@id="content-section"]/section[1]/div/div[4]/ul/li[1]/a').click()

您可以按照另一個@Andersson的建議使用 css 選擇器，而不是使用 xpath。

Answer 3

下一個按鈕往往因網頁而異...您必須檢查按鈕並使用 xpath 或 beaufifulsoup 對其進行處理

通常有“下一頁”和“上一頁”...將您的 xpath 定位到“下一頁”