分页和单击上的 Python Selenium 循环

Question

I am trying to find a solution with Python and selenium to scrap the data of a website. On the screenshot bellow the highest pagination is '40'. But next scrapping, it can be any value over 1.

# find Highest number in data-test-pagination-page-btn
s = # How can i get the highest pagination value with Selenium? 
print(s)
total_pages = int(s)

# How can I make python click on the pagination button 1,2,3 until S value?
for p in range (total_pages):
    driver.find_element_by_xpath("//ul//li[@xxxxxxxxxxxxxxxxxx']").click()
    time.sleep(3)

Is it possible to do this with selenium and python?

Answer 1

I am sure there is an easier solution that includes the attributes of the last element but since I don't know which website we're looking at, I am going to post the universal solution:

parent = driver.find_element_by_id("ember334")
children = parent.find_elements_by_tag_name("li")  # note the plural

for child in children:
    last_page = child.get_attribute("data-test-pagination-btn")
s = last_page

Since it overwrites with every iteration the last_page , only the last children with the highest pagination counts. A selenium object is not subscriptable, therefore you cant do s = children[-1].get_attribute("data-test-pagination-btn")