Python script dow download JSON data and save as/convert to CSV

Question

I am having trouble downloading data in a JSON format and converting it to a csv-file. In this case it is company overview data from alphavantage( https://www.alphavantage.co/query?function=OVERVIEW&symbol=IBM&apikey=demo )

I can download the data with the following script:

import requests
import pathlib

solditems = requests.get('https://www.alphavantage.co/query?function=OVERVIEW&symbol=IBM&apikey=demo') # (your url)
pathlib.Path('data2.json').write_bytes(solditems.content)

Then I have to data on my local drive in the json format. But then I try to convert to JSON into a CSV and so far different methods have failed for me. I tried this method with pandas ( https://datatofish.com/json-string-to-csv-python/ ) or this one ( https://medium.com/@gabrielpires/how-to-convert-a-json-file-to-csv-python-script-a9ff0a3f906e ) but I cant get any method to work. What would be the best way? I want a CSV-File with 2 lines. In the first line would be the keys like Symbol, Name, Decritpion etc and in the second line the values.

I just need a basic script to download JSON data and convert it to a CSV file. So I can parametrize it and build a loop around this process. But I am stuck

Answer 1

Pandas does have a json_normalize method to do that. Or, the other option is you can construct the dataframe from json (dictionaries) as long as each dictionary is within a list, where each row in your table is represented by each element in the list.

Also note, you can use .json() on the request to store right away. No need to write to file localay and then convert to csv.

import requests
import pandas as pd

solditems = requests.get('https://www.alphavantage.co/query?function=OVERVIEW&symbol=IBM&apikey=demo').json() # (your url)

df = pd.json_normalize(solditems)
df.to_csv('data.csv', index=False) 

Or with the single dictionary:

df = pd.DataFrame([solditems])

Output:

print(df)
  Symbol     AssetType  ... LastSplitFactor LastSplitDate
0    IBM  Common Stock  ...             2:1    1999-05-27

[1 rows x 60 columns]