Python Webscraping：需要帮助从 span html 标签获取数据值

Question

我目前正在尝试获取四舍五入美元的数据测试值，但我不断收到错误 NoneType，我只是想知道如何解决这个问题，谢谢 :D我想抓取的部分的网站 html 代码片段

这是我当前的代码：

import requests
from bs4 import BeautifulSoup

url = 'https://www.priceline.com/m/fly/search/YYZ-YUL-20200214/?cabin-class=ECO&no-date-search=false&search-type=11&num-adults=1&refclickid=https%3A%2F%2Fwww.google.com%2F'

page = requests.get(url)
soup = BeautifulSoup(page.content, 'html.parser')
price = soup.find('span', attrs={'data-test': 'rounded-dollars'})
print(price)

Answer 1

您将需要 selenium 来获取页面上显示的确切值，因为此页面的数据使用JSON Javascript动态填充。

from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path=r"***YOUR_CHROME-DRIVER_PATH***", chrome_options=chrome_options)
driver.get('https://www.priceline.com/m/fly/search/YYZ-YUL-20200214/?cabin-class=ECO&no-date-search=false&search-type=11&num-adults=1&refclickid=https%3A%2F%2Fwww.google.com%2F')

soup = BeautifulSoup(driver.page_source, 'lxml')
price = soup.find_all('span', {'data-test': 'rounded-dollars'})
for price in prices: print(price.text, end=' | ')

输出：

112 | 112 | 112 | 112 | 112 | 112