Parsing CSV to python dictionary using pandas

Question

I have following DataFrame parsed into Python:

df = pd.read_csv("my_file.csv")

result:

   indexes X_values Y_values
0       IDX1     x1      y1
1       IDX1     x2      y2
2       IDX1     x3      y3
3       IDX1     x4      y4
6       IDX2     x1      y1
9       IDX2     x4      y4
10      IDX3     x1      y1
11      IDX3     x2      y2

I need to create dictionaries with each indexes as key and x_values & y_values as list of nested dictionaries. Output should be like:

{"IDX1" : [{"x1": "y1"}, {"x2": "y2"}, {"x3": "y3"}, {"x4": "y4"}],
"IDX2": [{"x1": "y1"},{"x4": "y4"}],
"IDX3":[{"x1": "y1"}, {"x2": "y2"}]}

I am trying to parse it using set_index() method but always missing something. Could you help me?

Also, dictionary of nested dictionaries with indexes as keys would be also good solution.

Answer 1

We can do

d = df[['X_values','Y_values']].apply(lambda x : {x[0]:x[1]},axis=1).groupby(df['indexes']).agg(list).to_dict()
Out[104]: 
{'IDX1': [{'x1': 'y1'}, {'x2': 'y2'}, {'x3': 'y3'}, {'x4': 'y4'}],
 'IDX2': [{'x1': 'y1'}, {'x4': 'y4'}],
 'IDX3': [{'x1': 'y1'}, {'x2': 'y2'}]}

Answer 2

You can try

out = (df.apply(lambda row: {row['X_values']: row['Y_values']}, axis=1)
       .groupby(df['indexes']).agg(list).to_dict())

print(out)

{'IDX1': [{'x1': 'y1'}, {'x2': 'y2'}, {'x3': 'y3'}, {'x4': 'y4'}], 'IDX2': [{'x1': 'y1'}, {'x4': 'y4'}], 'IDX3': [{'x1': 'y1'}, {'x2': 'y2'}]}

Answer 3

Sounds like you need to make use of the to_dict() method ( documentation ).

You might need to index like you say. I would try:

df.set_index('indexes').T.to_dict('list')