将 JSON 转换为 Pandas Dataframe 在 Python

Question

There is data in json format as below:

dict = {"a":1,"b":2,"c":[{dic 1},{dic2},...so on]}

where dic 1 is defined below, like this list of dictionaries are there

dic 1 = {"d":4,"e":{"f":6,"g":7},"h":{"i":9,"j":[10,11,12]},"m":13}

so, the whole json file looks like the below:

dict = {"a":1,"b":2,"c":[{"d":4,"e":{"f":6,"g":7},"h":{"i":9,"j":[10,11,12]},"m":13},{dic2},...so on]}

Now I want to store this data as Pandas Dataframe like the below table, give your suggestion please

Expected Output:

Answer 1

The structure of your json is complex! Make it simple!

Your code won't run, returns unhashable type 'dict'. To solve, simply unpack any variable you're using in the main 'dict' (that's **dic1).

Even with that, you end with 2 rows and 3 columns. Why? The data in key 'c' is a list of dicts, pandas interpret list items as data for a column. Organize the json file.

Lastly, avoid using 'dict' to name a variable.

Answer 2

Try This

import json
import pandas as pd
from glob import glob
import matplotlib.pyplot as plt

#Convert json string to a flat python dictionary

def convert(x):
    ob = json.loads(x)
    for k, v in ob.copy().items():
        if isinstance(v, list):
            ob[k] = ','.join(v)
        elif isinstance(v, dict):
            for kk, vv in v.items():
                ob['%s_%s' % (k, kk)] = vv
            del ob[k]
    return ob

for json_filename in glob('*.json'):
    csv_filename = '%s.csv' % json_filename[:-5]
    print('Converting %s to %s' % (json_filename, csv_filename))
    df = pd.DataFrame([convert(line) for line in open(json_filename, encoding='utf-8')])
    df.to_csv(csv_filename, encoding='utf-8', index=False)

#Convert csv to pdf
data1 = pd.read_csv('data1.csv')
data2 = pd.read_csv('data2.csv')
data3 = pd.read_csv('data3csv')
data4 = pd.read_csv('data4.csv')
data5 = pd.read_csv('data5.csv')

https://gist.github.com/Elsaveram/3258db49eaac5e258401338ae17139a3

Answer 3

Save all the dictionary in file in format.json like this

import json
with open('dict.json', 'w') as fp:
json.dump(dict, fp,sort_keys=True, indent=4)

then try this

      df_json = pd.read_json(r'filepath\dict.json', lines=True)

if it didn't work we will figure out a regular expression to read the complexity inside. don't forget to import pandas and json, pandas can handle all the complexity most of the time. This will take much shorter time than using regular expressions and converting to csv files.