如何将多个列表添加到单个字典键而不合并它们？ 并转换为CSV

Question

I am trying to add multiple lists to a single dictionary key without merging them and convert to csv to get below expected result

list.txt contains [{A:1, B:2, C:3},{A:5, D:8, E:7, C:2}] [{A:12, B:23, C:100, D:23}]

then covert them into csv with fixed column header which should look like

Expected Result

A,B,C,D,E,F
1,2,3,,,
5,,2,8,7,
12,23,100,23,,

Here is my code -

csv_columns=A,B,C,D,E,F

listfromtextfile = eval(open("list").read())
def DictToCSV(csv_file,csv_columns,listIndex):
    try:
        with open(csv_file, 'ab') as csvfile:
            writer = csv.DictWriter(csvfile, fieldnames=csv_columns)
            HeadWrite = csv.writer(csvfile)
            HeadWrite.writerow(csv_columns)
            for data in listIndex:
                writer.writerow(data)
    except IOError as (errno, strerror):
            print("I/O error({0}): {1}".format(errno, strerror))
    return

for i in range(len(listfromtextfile)):
   DictToCSV(csv_file,csv_columns,listfromtextfile[i])

Giving last listfromtextfile[i] dictionary values in csv But here I am not able to find logic to create expected csv.

Answer 1

How do you add multiple lists to a single dictionary key without merging them?

There may be a more elegant way to do it, but is there any reason you can't establish the key value as a list and then wrap the items as you add them?

a = {'key': []}

y = [4, 5, 6]
z = [1, 2, 3]

a['key'] += [y]
a['key'] += [z]

# {'key': [[4, 5, 6], [1, 2, 3]]}

Answer 2

Suppose you have valid content in your source.txt :

[{"A":1, "B":2, "C":3},{"A":5, "D":8, "E":7, "C":2}]
[{"A":12, "B":23, "C":100, "D":23}]

- Option 1:

Read the file line by line, and convert to list, extend to one single list as output ( listfromtextfile ). Generate the headers which contain all the keys (remove duplicate one) from listfromtextfile . Generate the row_list which contain all the values as a list for per element in the listfromtextfile .

After that, you can use csv module to write the header row and values rows into your result.csv :

import csv
listfromtextfile =[]
for i in open("source.txt").readlines():
    listfromtextfile.extend(eval(i)) #generate the lsit from source file
headers = set([key for e in listfromtextfile for key in e.keys()]) #generate the headers for keys
row_list = [[e.get(header,'') for header in sorted(headers)] for e in listfromtextfile ] #generate the row_list for values
with open('result.csv', 'wb') as f:  # Just use 'w' mode in 3.x
    wr = csv.writer(f)
    wr.writerow(sorted(list(headers))) #write the header row
    wr.writerows(row_list) #write values rows

- Option 2:

Use pandas to generate the list of dict as dataframe, then call to_csv to write into result.csv :

import pandas as pd
listfromtextfile =[]
for i in open("source.txt").readlines():
    listfromtextfile.extend(eval(i)) #generate the lsit from source file
df = pd.DataFrame(listfromtextfile)
df.to_csv('result.csv',index=False)

The result.csv will be the one you want:

A,B,C,D,E
1,2,3,,
5,,2,8,7
12,23,100,23,

Answer 3

Taking reference from @Tiny.D solution, here is what I got working code in my python 2.6.x

import csv

listfromtextfile =[]
for i in open("source.txt").readlines():
    listfromtextfile.extend(eval(i)) #generate the lsit from source file
headers = set([key for e in listfromtextfile for key in e[0].keys()]) #generate the headers for keys
row_list = []
for e in listfromtextfile:
    for i in range(len(e)):
       row_list.append([e[i].get(header,'') for header in sorted(headers)]) #generate the row_list for values

with open('result.csv', 'wb') as f:  # Just use 'w' mode in 3.x
    wr = csv.writer(f)
    wr.writerow(sorted(list(headers))) #write the header row
    wr.writerows(row_list) #write values rows