How to convert a pandas DataFrame into a dictionary of lists with to_dict()?

Question

This appears to be a simple question, but I'm stuck by how tricky it is given the pandas.DataFrame.to_dict documentation:

I have the following toy example pandas DataFrame with two columns, whereby column2 is a column of lists:

import pandas as pd

dict1 = {'column1': ['list1', 'list2', 'list3'], 'column2': [[367, 30, 847, 482, 887, 654, 347, 504, 413, 821], [754, 915, 622, 149, 279, 192, 312, 203, 742, 846], [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]]}

df = pd.DataFrame(dict1)

print(df)
  column1                                            column2
0   list1  [367, 30, 847, 482, 887, 654, 347, 504, 413, 821]
1   list2  [754, 915, 622, 149, 279, 192, 312, 203, 742, 846]
2   list3  [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]

I would like to convert this pandas DataFrame into a dictionary, whereby each key is from column1 and the values are from column2 .

This is my preferred output:

{'list1': [367, 30, 847, 482, 887, 654, 347, 504, 413, 821], 
'list2': [754, 915, 622, 149, 279, 192, 312, 203, 742, 846], 
'list3': [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]}

Using .to_dict() , this appears very complicated though.

If I try df.set_index('column1').T.to_dict() , I get a dictionary with column names in the dictionary:

{'list1': {'column2': [367, 30, 847, 482, 887, 654, 347, 504, 413, 821]}, 
'list2': {'column2': [754, 915, 622, 149, 279, 192, 312, 203, 742, 846]}, 
'list3': {'column2': [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]}}

If I try with to_dict("list") , I get the following:

{'list1': [[367, 30, 847, 482, 887, 654, 347, 504, 413, 821]], 
'list2': [[754, 915, 622, 149, 279, 192, 312, 203, 742, 846]], 
'list3': [[586, 521, 470, 476, 693, 426, 746, 733, 528, 565]]}

which is incorrect, as now values of lists of lists, not a single list.

If I try to_dict("records") , the output is actually a single list, not a dictionary:

[{'list1': [367, 30, 847, 482, 887, 654, 347, 504, 413, 821], 
'list2': [754, 915, 622, 149, 279, 192, 312, 203, 742, 846], 
'list3': [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]}]

Is there another simple command in order to convert this pandas DataFrame into a dictionary of lists? I feel like I'm missing something.

Answer 1

Is this what you want:

df.set_index('column1')['column2'].to_dict()

Output:

{'list1': [367, 30, 847, 482, 887, 654, 347, 504, 413, 821],
 'list2': [754, 915, 622, 149, 279, 192, 312, 203, 742, 846],
 'list3': [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]}

Answer 2

You can use zip() :

>>> import pandas as pd
>>>
>>> dict1 = {'column1': ['list1', 'list2', 'list3'], 'column2': [[367, 30, 847, 482, 887, 654, 347, 504, 413, 821], [754, 915, 622, 149, 279, 192, 312, 203, 742, 846], [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]]}
>>>
>>> df = pd.DataFrame(dict1)
>>>
>>>
>>> ur_dict = dict(zip(df['column1'], df['column2']))
>>> ur_dict
{'list1': [367, 30, 847, 482, 887, 654, 347, 504, 413, 821], 'list2': [754, 915, 622, 149, 279, 192, 312, 203, 742, 846], 'list3': [586, 521, 470, 476, 693, 426, 746, 733, 528, 565]}
>>>