使用熊猫从URL读取导入的CSV文件时出错

Question

I'm a beginner trying to advance on a project that I learned with a tutorial. The project consists on importing a csv file from the United States Geological Survey and plotting its data on a map.

I managed to make it while using a file that's located in my computer. However, I cannot get around on getting the csv directly from the url, so that the data can update itself.

Right now I'm using pandas and getting this error:

File "C:/Users/Felipe/PycharmProjects/earthquake/earthquake.py", line 6, in with open(filename) as csvfile: TypeError: expected str, bytes or os.PathLike object, not DataFrame

import pandas as pd

filename = pd.read_csv('https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_hour.csv')
lats, lons = [], []

with open(filename) as f:
    reader = csv.reader(f)
    next(reader)
    for row in reader:
        lats.append(float(row[1]))
        lons.append(float(row[2]))

print('lats', lats[0:5])
print('lons', lons[0:5])

This part of the code is where I'm trying to retrieve de csv file.

I believe (again, I'm a beginner) that pandas actually gives me the data already "processed" (not as a csv anymore), and the tools I'm trying to use are usable on for csv files itself (this is me trying to understand the error, and I'm sure I'm probably wrong, but I couldn't think of anything else).

I tried looking at pandas documentations but couldn't find much infomation about "DataFrame" error. I also tried to use the ".to_csv" after importing the file, but it didn't work.

So, tl;dr how can I get around this code and import a csv file and use its data?

I also have some extra questions that I couldn't find the answer to: on "with open(filename) as f:" what does the "f" do?

Thanks a lot!

Answer 1

I believe the filename should just be the name.

filename = 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_hour.csv'

This should get it into a dataframe:

import io
import requests

content = requests.get(filename).content
df = pd.read_csv(io.StringIO(content.decode('utf-8')))

or simply:

df = pd.read_csv(filename)

Answer 2

pd.read_csv() also accepts URLs and returns parsed Pandas DataFrame.

Demo:

In [55]: url = 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/2.5_hour.csv'

In [56]: df = pd.read_csv(url)

In [57]: df
Out[57]:
                       time   latitude  longitude   depth   mag magType  nst  gap    dmin   rms    ...                      updated  \
0  2017-09-12T20:38:20.330Z -15.079900 -174.19000  144.26  4.80      mb  NaN   71  2.6100  0.85    ...     2017-09-12T20:56:45.040Z
1  2017-09-12T20:22:02.350Z  39.878502 -121.27433    6.78  2.74      md  9.0  215  0.3287  0.14    ...     2017-09-12T20:36:02.917Z

                             place        type horizontalError depthError  magError  magNst     status  locationSource magSource
0       107km NNW of Hihifo, Tonga  earthquake            8.80       5.30     0.033     285   reviewed              us        us
1  27km ENE of Magalia, California  earthquake            1.82      10.49     0.240       8  automatic              nc        nc

[2 rows x 22 columns]

In [58]: df[['latitude','longitude']]
Out[58]:
    latitude  longitude
0 -15.079900 -174.19000
1  39.878502 -121.27433