Python - Pandas : how to save csv file from url

Question

so I'm trying to get a csv file with requests and save it to my project:

import requests
import pandas as pd
import csv

def get_and_save_countries():
    url = 'https://www.trackcorona.live/api/countries'
    r = requests.get(url)
    data = r.json()
    data = data["data"]
  
    with open("corona/dash_apps/finished_apps/apicountries.csv","w",newline="") as f:  
        title = "location,country_code,latitude,longitude,confirmed,dead,recovered,updated".split(",") 
        cw = csv.DictWriter(f,title,delimiter=',', quotechar='|', quoting=csv.QUOTE_MINIMAL)
        cw.writeheader()
        cw.writerows(data)

I've managed that but when I try this:

get_data.get_and_save_countries()
df = pd.read_csv("corona\\dash_apps\\finished_apps\\apicountries.csv")

I get this error:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

And I have no idea why. Any help is welcome. Thanks.

Answer 1

Try:

 with open("corona/dash_apps/finished_apps/apicountries.csv","w",newline="", encoding ='utf-8') as f:

to explicitly specify the encoding with encoding='utf-8'

Answer 2

When you write to a file, the default encoding is locale.getpreferredencoding(False) . On Windows that is usually not UTF-8 and even on Linux the terminal could be configured other than UTF-8. Pandas is defaulting to utf-8 , so specify encoding='utf8' as another parameter to open .