使用 Selenium 和 Python，如何单击“查看更多”以在页面中加载更多项目？

Question

I preface this question with I am VERY new to learning Python, so this may be something basic and simple. I've done a lot of research and tried various approaches based on what has been provided as potential solutions to other questions, and I am stuck. I am trying to scrape data from different urls (for this specific example, https://www.dtlr.com/collections/men-footwear ), but when I scroll down the page there is a button that says "See More". As you can see in my script, I've tried various ways to click this button and continue the line of script to scroll through the newly listed items, but nothing seems to work. One post that I found mentioned that the button that needs to be clicked must be in view. I even tried scrolling back up using elem.send_keys(Keys.PAGE_UP) , and even with the button in view the clicking never worked. Any guidance?

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import re
import math
import random
import time

shoe_sites = ["https://www.dtlr.com/collections/men-footwear",
              "https://www.dtlr.com/collections/women-footwear",
              "https://www.dtlr.com/collections/kids-age-group-grade-school",
              "https://www.dtlr.com/collections/kids-age-group-pre-school",
              "https://www.dtlr.com/collections/kids-age-group-toddler-infant"]

for x in shoe_sites:
    my_url = x
    browser = webdriver.Chrome()
    browser.get(my_url)
    browser.maximize_window()
    time.sleep(random.randint(1,5))

    elem = browser.find_element_by_tag_name("body")

    no_of_pagedowns = 50

    while no_of_pagedowns:
        elem.send_keys(Keys.PAGE_DOWN)
        try:
            browser.find_element_by_css_selector('#ltkpopup-close-button > a').click()
        except Exception:
            try:
                browser.find_element_by_css_selector('#Collection > div > ul > a').click()
            except Exception:
                try:
                    browser.find_element_by_xpath('//*[@id="Collection"]/div/ul/a').click()
                except Exception:
                    try:
                        browser.find_elements_by_class_name('loadmore btn').click()
                    except Exception:
                        try:
                            element = browser.find_elements_by_class_name('loadmore btn')
                            coordinates = element.location_once_scrolled_into_view
                            browser.execute_script('window.scrollTo({}, {});'.format(coordinates['x'], coordinates['y']))
                            element.click()
                        except Exception:
                            test = 1

        time.sleep(random.randint(1,5))
        no_of_pagedowns-=1

    post_elems = browser.find_elements_by_class_name("product_men")
    html = browser.page_source
    browser.close()

Answer 1

To locate See More button you mean, use .find_element_by_css_selector("a.loadmore.btn")