[英]Cannot extract correct element in chrome dev tools when using Python
I am attempting to access the dates from this site with css selectors but it is not allowing me.我正在尝试使用 css 选择器从该站点访问日期,但它不允许我。 I keep getting this error: AttributeError: 'NoneType' object has no attribute 'select'我不断收到此错误: AttributeError: 'NoneType' object has no attribute 'select'
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.accuweather.com/en/us/san-
antonio/78205/daily-weather-forecast/351198")
soup = BeautifulSoup(page.content, 'html.parser')
daily = soup.find(class_="content-module")
period_tags = daily.select(".date .dow")
periods = [pt.get_text() for pt in period_tags]
periods
I expect the output to give me each of the days on the webpage in list form我希望 output 以列表形式在网页上的每一天给我
I only needed the one User-Agent header.我只需要一个用户代理 header。 However, content is dynamically generated so your requests response html won't be the same as on web-page where javascript can run.但是,内容是动态生成的,因此您的请求响应 html 不会与 javascript 可以运行的网页上的相同。 You can extract the required info from a script
tag in the response using regex and then parse with json parser您可以使用正则表达式从响应中的script
标记中提取所需信息,然后使用 json 解析器进行解析
import requests, re , json
headers = {'User-Agent': 'Mozilla/5.0'}
r = requests.get('https://www.accuweather.com/en/us/san-%20antonio/78205/daily-weather-forecast/351198', headers=headers)
p = re.compile(r'var dailyForecast = (.*);')
data = json.loads(p.findall(r.text)[0])
#print(data)
forecasts = {i['dow'] + ' - ' + i['date']:i['day'] for i in data}
print(forecasts)
dows = [i['dow'] for i in data]
print(dows)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.