[英]Scrape data from website to pandas dataframe
I'm trying to scrape weather data from the met office website, but I keep getting errors. 我正在尝试从会议办公室的网站上抓取天气数据,但是我一直遇到错误。
Here is what I have tried so far 到目前为止,这是我尝试过的
stats = ['Tmax', 'Tmin', 'Rainfall']
regions = ['England', 'Wales', 'Scotland']
base_url = r'https://www.metoffice.gov.uk/pub/data/weather/uk/climate/datasets/{}/date/{}.txt'
dframes = []
for r in regions:
for s in stats:
url = base_url.format(s,r)
df = pd.read_table(requests.get(url).content)
dframes.append(df)
I'm gettin an error: 我开始出错:
"OSError: Expected file path name or file-like object, got <class 'bytes'> type"
You have to use that bytes through a stream for example io.BytesIO
: 您必须通过流使用该字节,例如io.BytesIO
:
import pandas as pd
import requests
import io
stats = ['Tmax', 'Tmin', 'Rainfall']
regions = ['England', 'Wales', 'Scotland']
base_url = r'https://www.metoffice.gov.uk/pub/data/weather/uk/climate/datasets/{}/date/{}.txt'
dframes = []
for r in regions:
for s in stats:
url = base_url.format(s,r)
df = pd.read_table(io.BytesIO(requests.get(url).content))
dframes.append(df)
print(dframes)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.