Pandas CSV: csv to orderedDict

Question

What is the best way to save ordered dictionary to csv, load the csv file into ordered dictionary of same format, edit them and save them back.

I save my data originally as follows:

gifs_info_list=[]
gifs_info_list.append(OrderedDict([('ID','aswd'),('class','c1'),('mu',gif["parameters"]["mu"]),('sigma','23'), ('tags', [u'movies', u'hitler']),('flag1',False),('flag2',False),('flag3',False)]))

here is how I am doing orderedDict2CSV

def list_to_csv(mlist, fname):
    '''
    double check this method => where is the header added to the csv?
    Arguments
        mlist: A list of orderedDict objects.
        fname:
    '''
    df = pd.DataFrame(mlist)
    df.to_csv(fname, sep='\t')

This saves the data to csv, as follows:

  ID    class   mu  sigma   tags    flag1 flag2 flag3
0   fdex    c1  39.8348689258   2.49703182691   [u'movies', u'hitler']  False   False   False
2   11AE    c2  38.7252538197   2.23352853216   [u'movie', u'mark millar']  False   False   False
3   p9ut    c1  37.775785901    2.33519338409   []  False   False   False

I need to write CSV2OrderedDict that would load the data from the csv into a list of orderedDicts of the same format as it was created. How can I do this?

Answer 1

You can read csv file using pd.read_csv function. Then you can iterate over rows of DataFrame using iterrows() method, in each iteration create OrderedDict and append it to list.

import pandas as pd
from collections import OrderedDict
df = pd.read_csv("data.csv", sep="\t", index_col=0)
dicts = list()
for i, row in df.iterrows():
    d = OrderedDict(zip(row.index.tolist(),
                        row.tolist()))
    dicts.append(d)

The output will be:

>>> dicts

[OrderedDict([('ID', 'fdex'),
              ('class', 'c1'),
              ('mu', 39.8348689258),
              ('sigma', 2.49703182691),
              ('tags', "[u'movies', u'hitler']"),
              ('flag1', False),
              ('flag2', False),
              ('flag3', False)]),
 OrderedDict([('ID', '11AE'),
              ('class', 'c2'),
              ('mu', 38.7252538197),
              ('sigma', 2.23352853216),
              ('tags', "[u'movie', u'mark millar']"),
              ('flag1', False),
              ('flag2', False),
              ('flag3', False)]),
 OrderedDict([('ID', 'p9ut'),
              ('class', 'c1'),
              ('mu', 37.775785901),
              ('sigma', 2.33519338409),
              ('tags', '[]'),
              ('flag1', False),
              ('flag2', False),
              ('flag3', False)])]

EDIT: tags will be a string, not a list. If you want to convert it to list use

import ast
df['tags'] = df['tags'].apply(lambda x: ast.literal_eval(x))

Answer 2

def csv_to_list(fname):
    df = pd.from_csv(fname, index_col=0)
    return [OrderedDict(row[1]) for row in df.iterrows()]