find_elements_by_xpath 有时会返回一个员工列表？

Question

So I'm working on a program that will scrape information from Grailed ( https://www.grailed.com/ ), and my problem is that sometimes my call to selenium will return the data I want, but sometimes it returns an empty list. I'm new to Selenium and just trying to get a hang on how it works, I can't find a pattern to when it returns an empty list so I'm kind of stumped.

from selenium import webdriver

driver = webdriver.Chrome("Path to Chromedriver")
driver.get('https://www.grailed.com/designers/jordan-brand/hi-top-sneakers')

item = driver.find_elements_by_xpath('//html/body/div[3]/div[6]/div[3]/div[3]/div[2]/div[2]/div[1]/a/div[3]/div[1]/p[1]')
print(item)

One thing I would think is that the item specified by the xpath changes when you load the page again, but the xpath stays the same so I don't think that's it. Does anyone know why this is inconsistent in it's return, and how I can fix it?

Answer 1

Induce WebDriverWait () and wait for visibility_of_all_elements_located () and below xpath.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

driver=webdriver.Chrome("Path to Chromedriver")
driver.get("https://www.grailed.com/designers/jordan-brand/hi-top-sneakers")
elements=WebDriverWait(driver,10).until(EC.visibility_of_all_elements_located((By.XPATH,"//div[@class='feed-item']//div[@class='listing-designer-and-size']/p[@class='listing-designer truncate']")))
itemlist=[item.text for item in elements ]
print(itemlist)

Output :

['JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE', 'JORDAN BRAND × NIKE', 'JORDAN BRAND', 'JORDAN BRAND', 'JORDAN BRAND × NIKE × VINTAGE', 'JORDAN BRAND', 'JORDAN BRAND']