使用漂亮的汤刮取数据

Question

我对 web 抓取/python 相当陌生，并且有一段代码需要一些帮助。 我一辈子都看不到我哪里出了问题。 一如既往的帮助非常感谢。 如果你有时间，请指出我哪里出错了！

import requests
from bs4 import BeautifulSoup
import csv

page = requests.get("https://www.bathroomspareparts.co.uk/merlyn-two-panel-hinged-bath-screen-mb7-spare-parts-17909-c.asp")
soup = BeautifulSoup(page.content, 'html.parser')


all_products = []


products = soup.select('div.row cf')
for product in products:
    name = product.select('div.item-short-description')[0].text.strip() 
    price = product.select('div.item-price')[0].text.strip() 

    all_products.append({
        "Name": name,
        "Price": price,
    })

Answer 1

有些产品没有价格，因此您必须检查：

import csv
import requests
from bs4 import BeautifulSoup

page = requests.get(
    "https://www.bathroomspareparts.co.uk/merlyn-two-panel-hinged-bath-screen-mb7-spare-parts-17909-c.asp"
)
soup = BeautifulSoup(page.content, "html.parser")

all_products = []

products = soup.select("#products section")
for product in products:
    name = product.select_one(".item-short-description").text.strip()

    price = product.select_one(".item-price")
    price = price.text.strip() if price else "N/A"

    all_products.append(
        {
            "Name": name,
            "Price": price,
        }
    )

print(all_products)

印刷：

[
    {"Name": "Merlyn Bath Screen Cover Caps M7012", "Price": "£1.24"},
    {"Name": "Merlyn Luna Rail End Cap LH SP0M7010/L", "Price": "£1.81"},
    {"Name": "Merlyn Luna Rail End Cap RH SP0M7010/R", "Price": "£1.81"},
    {"Name": "Merlyn Luna Rail SP0M7005", "Price": "£6.54"},
    {
        "Name": "Merlyn Two Panel Hinged Bath Screen MB7 Spare Parts",
        "Price": "N/A",
    },
]