在 Pandas 中将数据帧列表导出和导入为 json 文件

Question

Pandas has the DataFrame.to_json and pd.read_json functions that work for single Data Frames. However, I have been trying to figure a way to export and import a list with many Data Frames into and from a single json file. So far, I have come to successfully export the list with this code:

with open('my_file.json', 'w') as outfile:
    outfile.writelines([json.dumps(df.to_dict()) for df in list_of_df])

This creates a json file with all the Data Frames converted to dicts. However, when I try to do the reverse to read the file and extract my Data Frames, I get an error. This is the code:

with open('my_file.json', 'r') as outfile:
    list_of_df = [pd.DataFrame.from_dict(json.loads(item)) for item in 
    outfile]

The error I get is: JSONDecodeError: Extra data

I think the problem is that I have to include somehow the opposite of 'writelines', which is 'readlines' in the code that reads the json file, but I do not know how to do it. Any help will be appreciated!

Answer 1

By using writelines your data isn't really a list in the python sense, which makes reading it a bit tricky. I'd recommend instead writing to your file like this:

with open('my_file.json', 'w') as outfile:
    outfile.write(json.dumps([df.to_dict() for df in list_of_df]))

Which means we can read it back just as simply using:

with open('my_file.json', 'r') as outfile:
    list_of_df = [pd.DataFrame.from_dict(item) for item in json.loads(outfile.read())]