[英]BeautifulSoup can't find my table because of some weird string
嗨,我一直在尝试使用 BeautifulSoup 从 wunderground 获取一张桌子,但它不起作用。
我认为它可能是表格标题旁边的星形字符串,但我无法修复它。
这是我的代码:
from bs4 import BeautifulSoup
import requests
url='https://www.wunderground.com/history/daily/LEMD/date/2020-10-21'
html_content = requests.get(url).text
soup = BeautifulSoup(html_content, "html.parser")
table = soup.find("table", {"class": "mat-table cdk-table mat-sort ng-star-inserted"})
table_data = table.tbody.find_all("tr")
和错误:
Traceback (most recent call last):
File "weather_poc.py", line 12, in <module>
table_data = table.tbody.find_all("tr")
AttributeError: 'NoneType' object has no attribute 'tbody'
您看到的数据是通过 JavaScript 从外部 URL 加载的。 您可以使用requests
/ json
模块来加载它。 例如:
import json
import requests
import pandas as pd
url = 'https://api.weather.com/v1/location/LEMD:9:ES/observations/historical.json?apiKey=6532d6454b8aa370768e63d6ba5a832e&units=e&startDate=20201021&endDate=20201021'
data = requests.get(url).json()
# uncomment this line to print all data:
# print(json.dumps(data, indent=4))
df = pd.json_normalize(data['observations'])
df.to_csv('data.csv', index=False)
创建data.csv
(来自 LibreOffice 的截图):
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.