Parsing JSON into pandas dataframe in Python 3

Question

I've been some trouble getting JSON code into a pandas dataframe in python.This is what my JSON code looks like:

    {
    "results": [
        {
            "events": [
                {
                    "id": 132,
                    "name": "rob",
                    "city": "nyc",
                    "age": 55
                },
                {
                    "id": 324,
                    "name": "sam",
                    "city": "boston",
                    "age": 35,
                    "favColor": "green"
                },
                                {
                    "id": 556,
                    "name": "paul",
                    "age": 23,
                    "favColor": "blue"
                },
                                {
                    "id": 635,
                    "name": "kyle",
                    "city": "nyc"
                }
            ]
        }
    ],
    "responseinfo": {
        "inspectedCount": 295822,
        "omittedCount": 0,
        "matchCount": 119506,
        "wallClockTime": 34
    }
}

I'm only trying to create a dataframe out of the data inside the events node and create columns of the keys. In some of these keys are missing however, so these would all have to be merged together to make sure all keys/columns exist. I tried cycling through each node populating a dictionary and then merging these but I cant figure it out. Any ideas how I can tackle this? Thanks! Rob

Answer 1

You can try to use the json module from the standard library to parse the json data, then converting the list of dicts to a Dataframe , like this:

import json
import pandas as pd

json_data = """ {
    "results": [
        { ..."""
 

data = json.loads(json_data)
events = data["results"][0]["events"]

df = pd.DataFrame(events)