Python3 beautifulsoup模块“ NoneType”错误

Question

I'm new to beautifulsoup module and I have a problem. My code is simple. Before all, the site I'm trying to scrape from is this and I am trying to scrape the price. (The big number two (2) with more of it)

My code:

import urllib
from bs4 import BeautifulSoup


quote_page = 'https://www.bloomberg.com/quote/SPX:IND'

page = urllib.request.urlopen(quote_page)

soup = BeautifulSoup(page, 'html.parser')

price_box = soup.find('div', attr = {'class': 'price'})
price = price_box.text

print(price)

The error I get:

price = price_box.text

AttributeError: 'NoneType' object has no attribute 'text'

Answer 1

I have used a more robust CSS Selector instead of the find methods. Since there is only one div element with class price , I am guessing this is the right element.

import requests
from bs4 import BeautifulSoup

response = requests.get('https://www.bloomberg.com/quote/SPX:IND')
soup = BeautifulSoup(response.content, 'lxml')
price = soup.select_one('.price').text
print(price)

Answer 2

Another solution:

from bs4 import BeautifulSoup
from requests import Session

session = Session()
session.headers['user-agent'] = (
    'Mozilla/5.0 (Windows NT 10.0; Win64; x64) '
    'AppleWebKit/537.36 (KHTML, like Gecko) Chrome/'
    '66.0.3359.181 Safari/537.36'
)

quote_page = 'https://www.bloomberg.com/quote/SPX:IND'

page= session.get(quote_page)

soup = BeautifulSoup(page.text, 'html.parser')

price_box = soup.find('meta', itemprop="price")

price = float(price_box['content'])

print(price)