Extract data from json using pandas in python

Question

I am new to programming. I am currently learning python. I have a question on extracting data from json using pandas in python. I have a COVID data set and I want to visualize it using python. Here is the data set:

[
    {
        "Confirmed": 0,
        "Country/Region": "Afghanistan",
        "Date": "2020-01-22",
        "Deaths": 0,
        "Province/State": null,
        "Recovered": 0
    },
    {
        "Confirmed": 0,
        "Country/Region": "Afghanistan",
        "Date": "2020-01-23",
        "Deaths": 0,
        "Province/State": null,
        "Recovered": 0
     }
]

And here is my code:

import json

import pandas as pd


all_covid_data = pd.read_json(path+'Data/readable_covid_data.json', lines=False, orient="records")


print(all_covid_data)

It raised an error said,

loads(json, precise_float=self.precise_float), dtype=True

ValueError: Expected object or value

Does anyone have an idea on this?

Answer 1

I think it might be due to entering the path to the json file incorrectly. Please check if you're running the python file from the correct directory.

Or maybe you could enter the complete path for your json file just for checking the code if this seems confusing.

Answer 2

I think there are two issues here. One is related to Path and another is related to dtype . For resolving these issues kindly refer to the code given below:-

# Import all Important Libraries
import json
import pandas as pd
import datetime

# Use 'Absolute Path' instead of 'Normal Path' Because it fetch Data from Root Directory. 
# 'Absolute Path' example:- "C://Project//Data//"
# 'Normal Path' OR 'Relative Path' example:- "/Data/"

# Removed 'Data/' Because we have Used Absolute Path and define 'dtype' of each column 
all_covid_data = pd.read_json('E://Stackoverflow//readable_covid_data.json', lines=False, orient="records",convert_dates=['Date'] ,dtype={"Confirmed":int, "Country/Region":str, "Deaths":int, "Province/State":str, "Recovered":int})

# Print few Records of 'COVID Dataset'
all_covid_data.head()

# Output of Above Cell:-

    Confirmed   Country/Region  Date        Deaths  Province/State  Recovered
0   0           Afghanistan     2020-01-22  0       None            0
1   0           Afghanistan     2020-01-23  0       None            0

Hope this Solution helps you.

Answer 3

The example code written does not provide a value for path so this would throw an error as the code cannot find the file.

Here is a working example (for windows10 operating system).

import json
import pandas as pd

# put in the full path to the file to be read.
file_to_read = r'C:\Users\Owner\Dropbox\programming\python\general\tests\simple_json.json'

val = pd.read_json(file_to_read)


print(val)

And this is the result of the print:

   Confirmed Country/Region       Date  Deaths  Province/State  Recovered
0          0    Afghanistan 2020-01-22       0             NaN          0
1          0    Afghanistan 2020-01-23       0             NaN          0