将带有嵌套对象的JSON转换为Pandas Dataframe

Question

I am trying to load json from a url and convert to a Pandas dataframe, so that the dataframe would look like the sample below.

I've tried json_normalize, but it duplicates the columns, one for each data type (value and stringValue). Is there a simpler way than this method and then dropping and renaming columns after creating the dataframe? I want to keep the stringValue.

    Person ID   Position ID     Job ID  Manager
0   192         936             93      Tom



my_json = {

    "columns": [
        {
            "alias": "c3",
            "label": "Person ID",
            "dataType": "integer"
        },
        {
            "alias": "c36",
            "label": "Position ID",
            "dataType": "string"
        },
        {
            "alias": "c40",
            "label": "Job ID",
            "dataType": "integer",
            "entityType": "job"
        },
        {
            "alias": "c19",
            "label": "Manager",
            "dataType": "integer"
        },
     ],
    "data": [
        {
            "c3": {
                "value": 192,
                "stringValue": "192"
            },
            "c36": {
                "value": "936",
                "stringValue": "936"
            },
            "c40": {
                "value": 93,
                "stringValue": "93"
            },
            "c19": {
                "value": 12412453,
                "stringValue": "Tom"
            }
        }
    ]
}

Answer 1

If c19 is of type string, this should work

alias_to_label = {x['alias']: x['label'] for x in my_json["columns"]}
is_str = {x['alias']: ('string' == x['dataType']) for x in my_json["columns"]}

data = []
for x in my_json["data"]:
    data.append({
        k: v["stringValue" if is_str[k] else 'value']
        for k, v in x.items()
    })
df = pd.DataFrame(data).rename(columns=alias_to_label)