BeautifulSoup AttributeError: 'NoneType' 对象没有属性 'text'

Question

I am trying to learn how to scrape websites and therefore not using an API. I am trying to scrape eBay's websites and my script runs for a few minutes but stops because of an error stating "AttributeError: 'NoneType' object has no attribute 'text'". I did my due diligence and search on Google/StackOverflow help but was unable to find any. The error points to the code below but I cannot figure out how to fix it.

total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

if total_sold_price:
    total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text

else:
    try:
        total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
    except Exception as e:
        print(e)
        total_sold_price = ""

Entire code: https://pastebin.com/xS4bAwZK

Thanks in advance, I greatly appreciate it.

Answer 1

The issue is soup.find('span', {'class': 'vi-qtyS-hot-red'}) returns None, and then you are trying to extract the text .text of None (hence the error message). There are a few ways you can handle it. All I did was move around some logic:

Also, you have it store the text total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text , and if it stores it, you have it do the exact something which is redundant.

Lastly, the page is dynamic, so you might want to look into API or other ways to access the data.

But for your code provided, maybe something like this instead:

try:
    total_sold_price = soup.find('span', {'class': 'vi-qtyS-hot-red'}).text
except Exception as e:
    try:
        total_sold_price = soup.find('a', {'class': 'vi-txt-underline'}).text
    except Exception as e:
                print(e)
                total_sold_price = ""

Answer 2

Is it solved? Instead of {'class': 'vi-qtyS-hot-red'} You can write = 'vi-qtyS-hot-red'

No need write 'class' and {}