![](/img/trans.png)
[英]How to Group Json data based on Key and combine values under in Python?
[英]Python: Combine data based on a key column
在同一個文本文件(兩個標題)中包含包含父記錄和子記錄的數據。 Parent 是department
,childs 是employees
, dno
是連接列。
dno,dname,loc
10,FIN,USA
20,HR,EUR
30,SEC,AUS
empno,ename,sal,dno
2100,AMY,1001,10
2200,JENNY,2001,10
3100,RINI,3001,20
4100,EMP4,4001,30
4200,EMP5,5001,30
4300,EMP6,6001,30
想通過dno
組合這兩個數據並創建一個 output 如下所示:
empno,ename,sal,dno,dname,loc
2100,AMY,1001,10,FIN,USA
2200,JENNY,2001,10,FIN,USA
3100,RINI,3001,20,HR,EUR
4100,EMP4,4001,30,SEC,AUS
4200,EMP5,5001,30,SEC,AUS
4300,EMP6,6001,30,SEC,AUS
Python version - 2.6
已嘗試以下解決方案:
dept_lst = []
emp_lst = []
with open(efile,'rb') as e_file:
reader = csv.reader(e_file,delimiter=",")
for row in reader:
if ((row[0] != 'dno' and row[0] != 'dname' ) or
(row[0] != 'empno' and row[0] != 'ename')):
if len(row) == 3:
dept_lst.append(row)
elif len(row) == 4:
emp_lst.append(row)
result = [ e + [d[1],d[2]] for e in emp_lst for d in dept_lst if e[3] == d[0]]
for line in result:
print ",".join(line)
問題:原始數據超過 1GB,這似乎有效。 不確定這是否是最佳解決方案。
想知道使用Python Standard Library - 2.6
是否有任何其他有效的方法/替代方案來處理這種情況。
考慮閱讀第一部分並建立一個后續詞典,然后切換到第二部分並使用該詞典。 此外,考慮使用 CSV 編寫器一次寫入已處理的行,而不是將它們保存為列表。
dno = {}
# Why do you open the file in the binary mode?
with open("efile.csv", "r") as e_file,\
open("ofile.csv", "w") as o_file:
reader = csv.reader(e_file)
next(reader) # Skip the header
for row in reader:
if row[0] == 'empno':
break # The second part begins
dno[row[0]] = row[1:]
writer = csv.writer(o_file)
for row in reader:
writer.writerow(row + dno[row[3]])
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.