抓取網站時如何在Python Selenium中轉到下一頁直到最后一頁？

Question

圖像用於 CSS 選擇器，xpath 用於分頁。

我還想執行正則表達式以將 Apple、iPhone 12、Neo Galactic Silver 分開，我想在新行中打印它。

完成當前頁面的產品列表后，我希望能夠單擊下一步並與下一頁上的產品執行相同的過程。

這就是問題所在：當它到達當前頁面的10個項目時，我不知道如何切換到另一個頁面並重新開始。

import xlwt
from selenium import webdriver
import re
import time

class cometmobiles:
    def __init__(self):
        self.url='https://www.mediaworld.it/catalogo/telefonia/smartphone-e-cellulari/smartphone'
    def comet(self):
        try:
            driver=webdriver.Chrome()
            driver.get(self.url)
            time.sleep(5)
            cookies = driver.find_element_by_id("onetrust-accept-btn-handler")    
            cookies.click()
            print("accepted cookies")
            driver.maximize_window()
            print("window maximized")
            mylist = []
            hasNextPate = True
            while hasNextPate:
                containers = []
                containters =driver.find_elements_by_css_selector('article[class="product clearfix p-list-js"]')
                for container in containters:
                    #Title
                    try:
                        title = container.find_element_by_css_selector('h3[class="product-name"]').text
                        print(title)
                    except:
                        pass

                    #price
                    try:
                        price = container.find_element_by_css_selector('span[class="price mw-price enhanced"]').text
                        print(price)
                    except:               
                        pass                    
                try:
                    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
                    time.sleep(5)
                    nxt=driver.find_elements_by_css_selector('span[class="pages"] a')
                    time.sleep(5)
                    nxt.click()
                except:
                    break
        except:
            pass
comets=cometmobiles()
comets.comet()    

Answer 1

而不是這部分

try:
    driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
    time.sleep(5)
    nxt=driver.find_elements_by_css_selector('span[class="pages"] a')
    time.sleep(5)
    nxt.click()
except:
    break

您可以使用它，如果頁碼不存在，則網站打開主頁，因此您應該添加

try:
  x=0
  while True:
    x+=1
    driver.get(url+"?pageNumber="+str(x)) #Get the next page
    if driver.current_url == url: #If there is no next page it will turn main page and you can break at this time
      break
except:
  pass