Python3: Can't pickle DictReader instance

Question

I have a file csvHelper.py where I read the csv and store it in a dictionary using DictReader. But when I try to pickle this dictionary, I get the following error:

_pickle.PicklingError: Can't pickle : it's not the same object as _csv.reader

For reference, the relevant part of the code:

allData = DictReader(open('xyz.csv', 'rt'))

for row in allData:
    row["Element name"] = row["Element name"]+'##'

dataStore = open('myPickleFile', 'wb')
pickle.dump(allData, dataStore)

Answer 1

The code is not pickling the dictioary, but DictReader .

If you want dump all dictionaries, gather them in a list, and dump the list:

import pickle
from csv import DictReader

with open('xyz.csv', 'rt') as f:
    allData = DictReader(f)

    rows = []
    for row in allData:
        row["Element name"] = row["Element name"]+'##'
        rows.append(row)

with open('myPickleFile', 'wb') as f:
    pickle.dump(rows, f)

Answer 2

DictReader is indeed unpicklable. Try converting allData to a list prior to processing:

allData = list(DictReader(open('xyz.csv', 'rt')))

This would also ensure that the changes you do to the individuals rows while iterating aren't lost.