Read txt file directly from web in Python

Question

Is there a straightforward way to read txt file directly from web in Python? This is simple in R, if you want to read a table, you just specify the URL location, eg:

dat = read.csv(<URL/file.csv>)

Answer 1

You can read a CSV file via URL in Python using requests module then use csv module with either csv.reader() or csv.DictReader() to parse each row of the CSV file.

Example:

import csv
import io
import requests
    
url = 'https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_hour.csv'
r = requests.get(url)
buff = io.StringIO(r.text)
dr = csv.DictReader(buff)
for row in dr:
    print(row['time'], row['mag'], row['place'])

This prints out the time, mag and place fields of the CSV file.

2020-10-05T16:34:52.160Z 1.41 11km ENE of Ridgecrest, CA
2020-10-05T16:29:26.070Z 2.39 16km WSW of Toms Place, CA
2020-10-05T16:24:08.860Z 2.42 1 km SSE of Pāhala, Hawaii
...

You can also use pandas to parse the CSV data.

Answer 2

Here is an example of loading csv file using requests library:

import pandas as pd 
import requests
import io

# load file
data = requests.get('<<URL/file.csv>>').content

# decode data
data = data.decode('utf-8')

# load data into dataframe, optionally separator can be specified using sep="," or sep=";"
df = pd.read_csv(io.StringIO(data))