使用 python 从 pipe 分隔的平面文件创建嵌套的 json 行

Question

我有一个文本文件 pipe，分隔如下。 在该文件中，对于相同的 ID、CODE 和 NUM 组合，我们可以有不同的 INC 和 INC_DESC

ID|CODE|NUM|INC|INC_DESC
"F1"|"W1"|1|1001|"INC1001"
"F1"|"W1"|1|1002|"INC1002"
"F1"|"W1"|1|1003|"INC1003"
"F2"|"W1"|1|1002|"INC1003"
"F2"|"W1"|1|1003|"INC1004"
"F2"|"W2"|1|1003|"INC1003"

我们想像下面这样创建 json，其中不同的 INC 和 INC_DESC 应该作为 ID、CODE 和 NUM 的相同组合的数组出现

{"ID":"F1","CODE":"W1","NUM":1,"INC_DTL":[{"INC":1001, "INC_DESC":"INC1001"},{"INC":1002, "INC_DESC":"INC1002"},{"INC":1003, "INC_DESC":"INC1003"}]}
{"ID":"F2","CODE":"W1","NUM":1,"INC_DTL":[{"INC":1002, "INC_DESC":"INC1002"},{"INC":1003, "INC_DESC":"INC1003"}]}
{"ID":"F2","CODE":"W2","NUM":1,"INC_DTL":[{"INC":1003, "INC_DESC":"INC1003"}]}

我在下面尝试过，但它没有按照我的意愿生成嵌套

import pandas as pd

Input_File=f'V:\input.dat'
df=pd.read_csv(Input_File, sep='|')

json_output=f'V:\outfile.json'
output=df.to_json(json_output, orient='records')

Answer 1

import pandas as pd


# agg function
def agg_that(x):
    l = [x]
    return l


Input_File = f'V:\input.dat'
df = pd.read_csv(Input_File, sep='|')

# groupby columns
df = df.groupby(['ID', 'CODE', 'NUM']).agg(agg_that).reset_index()
# create new column
df['INC_DTL'] = df.apply(
    lambda x: [{'INC': inc, 'INC_DESC': dsc} for inc, dsc in zip(x['INC'][0], x['INC_DESC'][0])], axis=1)
# drop old columns
df.drop(['INC', 'INC_DESC'], axis=1, inplace=True)

json_output = f'V:\outfile.json'
output = df.to_json(json_output, orient='records', lines=True)

OUTPUT：

{"ID":"F1","CODE":"W1","NUM":1,"INC_DTL":[{"INC":1001,"INC_DESC":"INC1001"},{"INC":1002,"INC_DESC":"INC1002"},{"INC":1003,"INC_DESC":"INC1003"}]}
{"ID":"F1","CODE":"W2","NUM":1,"INC_DTL":[{"INC":1003,"INC_DESC":"INC1003"}]}
{"ID":"F2","CODE":"W1","NUM":1,"INC_DTL":[{"INC":1002,"INC_DESC":"INC1003"},{"INC":1003,"INC_DESC":"INC1004"}]}