如何最好地在 Python 中展平 NDJson 数据

Question

I have a huge file (>400MB) of NDJson formatted data and like to flatten it into a table format for further analysis.

I started iterate through the various objects manually but some are rather deep and might even change over time, so I was hoping for a more general approach.

I was certain pandas lib would offer something but could not find anything that would help my case. Also, the several other libs I found seem to not 'fully' provide what I was hoping for (flatten_json). It all seems very early on.

Is it possible that there is not good (fast and easy) solve for this at this time?

Any help is appreciated

Answer 1

pandas read_json有一个 bool param lines ，将其设置为 True 以读取 ndjsons

data_frame = pd.read_json('ndjson_file.json', lines=True)