试图 web 抓取动态网页，没有得到错误代码。 如何验证“它正在工作”？

Question

I'm trying to webscrape a dynamic website with the following code below. My first few attempts returned some error messages like "return self.find_element(by=By.CLASS_NAME, value=name)" and some "Message: no such element: Unable to locate element". However, now it has been working but hasn't given me the results I want. What element should I be targeting? Is it possible something is wrong with my setup of selenium and chromedriver(had trouble with these initially too).

   from selenium import webdriver
   url = 'https://www.landers.ph/beverages/coffee.html'
   driverlocation = '/Users/username/Downloads/chromedriverdriver = 
   webdriver.Chrome(driverlocation)
   driver.get(url)
   products = driver.find_elements_by_class_name('ld-product-detail-info')

   for product in products:
       name = product.find_element_by_class_name('productTitle').text
       price = product.find_element_by_class_name('priceComponent').text
       print(name,price)

Answer 1

Induce WebDriverWait () and wait for visibility_of_all_elements_located () and following class name

driver.get('https://www.landers.ph/beverages/coffee.html')
products=WebDriverWait(driver,20).until(EC.visibility_of_all_elements_located((By.CLASS_NAME,"productComponent")))
for product in products:
    name = product.find_element_by_class_name('productTitle').text
    price = product.find_element_by_class_name('priceComponent').text
    print(name,price)

Output :

Cafe Puro 100% Pure Instant Coffee in Mason Jar 90g ₱99.75
Cafe Puro 3-in-1 Instant Coffee 20 x 17g ₱85.75
San Mig Sugar Free White 3 in 1 Coffee 10 x 9g ₱61.75
Nescafe Classic Double Filter Instant Coffee Pack 200g ₱144.75
Nescafe Original Instant Coffee Mix 840g ₱171.75
Nescafe Creamy White 3-in-1 Instant Coffee 870 g ₱175.75
San Mig Super 3-in-1 Sugar Free Coffee 20 pcs x 7 g ₱111.75
Good Day Vanilla Latte 3 in 1 Instant Coffee 30pcs. x 20 g ₱139.75
Good Day Cappuccino with Coklat Granule 30 x 25g ₱199.75
Gold Choice Original Coffee Mix 20x 20g ₱149.75
Kopiko Black 3-in-1 Instant Coffee Mix 30 x 30g ₱209.25
San Mig Original 3-in-1 Coffee Mix 30 x 20g ₱159.75
Nescafe Gold Latte Macchiato 10 x 20g ₱169.75
Nescafe Classic Decaf Resealable Pack Instant Coffee 160 g ₱162.75
Cafe Puro Jade Blend Fresh Ground Coffee in a Tin Can 400g ₱188.75
Kopiko Brown Coffee Mix 30 x 27.5g ₱210.75
Pure Nature 12-in-1 Green Coffee Mix 10 x 21g ₱279.75
Kopiko Blanca Creamy Coffee Mix 30 x 30g ₱210.75
Folgers Classic Medium Roast Ground Coffee 320 g ₱349.75
Kopiko Cappuccino Coffee Mix 30 x 25g ₱210.75
Check Hup 3-in-1 Ipoh White Coffee King 15 x 40 g ₱318.75
Nescafe Taster's Choice French Roast Instant Coffee 198g ₱619.75
Nescafe Taster's Choice House Blend Decaf Coffee 7 oz ₱679.75
Maxwell House Cappuccino 750 g ₱899.75

You need to import below libraries.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By