csv.writerows writes lists as strings in the output CSV file

Question

I am trying to write a script that takes a dictionary whose items are lists, and write those lists to a CSV file. The code I have is as follows:

import csv

data = {'a': [1, 2, 3, 4], 'b': [5,6,7,8]}
shape = {'a': ['round', 'square'], 'b': ['triangle', 'oval']}
size = {'a': [100, 1000], 'b': [750, 750]}

with open('test.csv', 'w+') as f_raw:
    writer = csv.writer(f_raw, delimiter=',', lineterminator='\n')
    for id in data.keys():
        line = [id, data[id], shape[id], size[id]]
        writer.writerow(line)

The output CSV file takes the form of

a,"[1, 2, 3, 4]","['round', 'square']","[100, 1000]"
b,"[5, 6, 7, 8]","['triangle', 'oval']","[750, 750]"

All the lists are enclosed in quotation marks, making them a string. I wish for them to remain as lists so when I read them, python detects them as lists, not strings. What have I been missing that causes this issue?

I am using python 3.6

Answer 1

csv module writerow takes a sequence as input ( list or tuple are ok). For each item it converts them to their string representation, which is fine for integers & floats (and strings).

But for lists, it's not what you're expecting.

Solution: just create your row by adding the lists together:

with open('test.csv', 'w',newline="") as f_raw:
    writer = csv.writer(f_raw, delimiter=',', lineterminator='\n')
    for id in data.keys():
        line = [id] + data[id] + shape[id] + size[id]
        writer.writerow(line)

with your data, the file now contains:

b,5,6,7,8,triangle,oval,750,750
a,1,2,3,4,round,square,100,1000

Answer 2

This may not directly solve your problem but if you have a list like

a = ["[1, 2, 3, 4]","['round', 'square']","[100, 1000]"]

Then following will surely be handy:

from ast import literal_eval
>>> a = ["[1, 2, 3, 4]","['round', 'square']","[100, 1000]"]
>>> list(map(literal_eval, a))
[[1, 2, 3, 4], ['round', 'square'], [100, 1000]]

From literal_eval you can always convert strings back to their original form.

Answer 3

Also, you can simply unpack your lists:

for id in data.keys():
    line = [id, *data[id], *shape[id], *size[id]]
    writer.writerow(line)

result:

a,1,2,3,4,round,square,100,1000
b,5,6,7,8,triangle,oval,750,750

Of course, it is not grouped, but if you have strongly formatted data, it is easy to parse it back.