Python Selenium keep clicking next button

Question

I am trying to crawl multiple pages using Python Selenium from https://www.walmart.com/ip/Clorox-Disinfecting-Wipes-On-The-Go-Citrus-Blend-Scent-34-Wipes/29701960?page=seeAllReviews .

There are buttons at the bottom of the web page. HTML looks like below:

<ul class="paginator-list">
<li><button aria-label="Page 1 of 6 selected" class="active">1</button></li>
<li><button aria-label="Page 2 of 6 " class="">2</button></li>
<li><button aria-label="Page 3 of 6 " class="">3</button></li>
<li><button aria-label="Page 4 of 6 " class="">4</button></li>
<li><button aria-label="Page 5 of 6 " class="">5</button></li>
<li class="paginator-list-gap"></li>
<li><button aria-label="Page 3141 of 6 " class="">3141</button></li>
</ul>

How do I click on second button (Page 2 of 6) using Selenium? How do I keep clicking on next button as page changes. Any suggestions?

Answer 1

您可以每次使用此xpath单击nxt按钮链接： //button[@class="active"]/ancestor::li/following-sibling::li[1]

Answer 2

As per the HTML you have shared and your comment to click on 2nd button (Page 2 of 6) you have to scroll the pagination webelement within the Viewport and then invoke click() as follows :

from selenium import webdriver

driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://www.walmart.com/ip/Clorox-Disinfecting-Wipes-On-The-Go-Citrus-Blend-Scent-34-Wipes/29701960?page=seeAllReviews")
pagination_element = driver.find_element_by_xpath("//div[@class='ReviewsFooter-pagination arrange arrange-spaced']")
driver.execute_script("return arguments[0].scrollIntoView(true);", pagination_element)
driver.find_element_by_xpath("//div[@class='ReviewsFooter-pagination arrange arrange-spaced']//ul[@class='paginator-list']/li/button[@aria-label='Page 2 of 6 ']").click()
print("Clicked on Page 2")
driver.quit()

Console Output :

Clicked on Page 2