[英]flatten_json recursive flattening function for lists
我想在每个级别展平以下 JSON 并创建一个 pandas dataframe 每个级别,我使用flatten_json
通过每个级别循环创建多个嵌套循环:
{
"metadata": {
"name": "abc",
"time": "2020-04-01"
},
"data": [
{
"identifiers": [
{
"type": "abc",
"scheme": "def",
"value": "123"
},
{
"type": "abc",
"scheme": "def",
"value": "123"
}
],
"name": "qwer",
"type": "abd",
"level1": [
{
"identifiers": [
{
"type": "abc",
"scheme": "def",
"value": "123"
},
{
"type": "abc",
"scheme": "def",
"value": "123"
}
],
"name": "asd",
"type": "abd",
"level2": [
{
"identifiers": [
{
"type": "abc",
"scheme": "def",
"value": "123"
},
{
"type": "abc",
"scheme": "def",
"value": "123"
}
],
"name": "abs",
"type": "abd"
},
{
"identifiers": [
{
"type": "abc",
"scheme": "def",
"value": "123"
},
{
"type": "abc",
"scheme": "def",
"value": "123"
}
],
"name": "abs",
"type": "abd"
}
]
}
]
}
]
}
我正在尝试使用以下代码使用 flatten_json (在 Python 中展平 JSON )展平这个 json :
import pandas as pd
import flatten_json as fj
import json
level2 = []
keys = {'data', 'level1', 'level2'}
with open('test_lh.json') as f:
data = json.load(f)
for x in data['data']:
for y in x['level1']:
for z in y['level2']:
dic = fj.flatten(z)
level2.append(dic)
df = pd.DataFrame(level2)
print(df)
Output 给出如下:
identifiers_0_type identifiers_0_scheme identifiers_0_value identifiers_1_type identifiers_1_scheme identifiers_1_value name type
0 abc def 123 abc def 123 abs abd
1 abc def 123 abc def 123 abs abd
我将如何编写递归 function 以获得相同的 output 而无需调用 n 个 for 循环? 这些级别可以 go 下降多个级别。 我已经尝试为此使用json_normalize
但我还需要最终 output 中的父级标识符,并且json_normalize
不适用于多个记录路径。
我使用递归解决了它,这是我的代码:
import json
import pandas as pd
import flatten_json as fj
keys = {'data', 'level1', 'level2', 'level3'}
with open('test_lh.json') as f:
data = json.load(f)
levels = ['data.level1.level2.level3', 'data.level1.level2', 'data.level1', 'data']
recs_dict = {}
def do_step(data_dict, level, depth, path):
recs = []
for x in data_dict[level]:
if depth < len(path.split('.'))-1:
do_step(x, path.split('.')[depth+1], depth+1, path)
else:
dic = fj.flatten(x, root_keys_to_ignore=keys)
recs.append(dic)
recs_dict[level] = recs
for path in levels:
do_step(data, path.split('.')[0], 0, path)
for key, value in recs_dict.items():
print(key)
df = pd.DataFrame(recs_dict[key])
print(df)
这是 output:
level3
identifiers_0_type identifiers_0_scheme identifiers_0_value identifiers_1_type identifiers_1_scheme identifiers_1_value name type
0 abc def 123 abc def 123 abs level3
1 abc def 123 abc def 123 abs level3
level2
identifiers_0_type identifiers_0_scheme identifiers_0_value identifiers_1_type identifiers_1_scheme identifiers_1_value name type
0 abc def 123 abc def 123 abs level2
1 abc def 123 abc def 123 abs abd
level1
identifiers_0_type identifiers_0_scheme identifiers_0_value identifiers_1_type identifiers_1_scheme identifiers_1_value name type
0 abc def 123 abc def 123 asd level1
data
identifiers_0_type identifiers_0_scheme identifiers_0_value identifiers_1_type identifiers_1_scheme identifiers_1_value name type
0 abc def 123 abc def 123 qwer abd
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.