我正在尝试使用 BeautifulSoup 获取产品信息，但无法获取之间的文本<span> </span>

Question

def Bprice2(url):
    print("Fetching details....")
    response=requests.get(url)
    detail=response.content
    soup=BeautifulSoup(detail,'html.parser')
    name=soup.find('span',{'class':'a-size-medium a-color-base a-text-normal'}).text
    print(str(name))
    
URL2='https://www.amazon.in/s?k=iphone+11'
bestprice2=Bprice2(URL2)

this is the error

Traceback (most recent call last):
   File "c:\Users\91858\Desktop\py auto\bestprice.py", line 37, in <module>
    bestprice2=Bprice2(URL2)
   File "c:\Users\91858\Desktop\py auto\bestprice.py", line 32, in Bprice2
    name=soup.find('span',{'class':'a-size-medium a-color-base a-text-normal'}).text 
AttributeError: 'NoneType' object has no attribute 'text' 

Answer 1

Just add headers to you requests call so you will able to find data easily.

import requests
from  bs4 import BeautifulSoup
headers={"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36"}
res=requests.get("https://www.amazon.in/s?k=iphone+11",headers=headers)
soup=BeautifulSoup(res.text,"html.parser")
name=soup.find("span",class_="a-size-medium a-color-base a-text-normal").get_text()

Output:

'Apple iPhone 11 (128GB) - White'