Pandas groupby and get dict in list

Question

I'm trying to extract grouped row data to use values to plot it with label colors another file.

my dataframe is like below.

df = pd.DataFrame({'x': [1, 4, 5], 'y': [3, 2, 5], 'label': [1.0, 1.0, 2.0]})

    x   y   label
0   1   3   1.0
1   4   2   1.0
2   5   5   2.0

I want to get group of label list like

{'1.0': [{'index': 0, 'x': 1, 'y': 3}, {'index': 1, 'x': 4, 'y': 2}],
 '2.0': [{'index': 2, 'x': 5, 'y': 5}]}

How to do this?

Answer 1

df = pd.DataFrame({'x': [1, 4, 5], 'y': [3, 2, 5], 'label': [1.0, 1.0, 2.0]})
df['index'] = df.index
df
   label  x  y  index
0    1.0  1  3      0
1    1.0  4  2      1
2    2.0  5  5      2

df['dict']=df[['x','y','index']].to_dict("records")
df
   label  x  y  index                             dict
0    1.0  1  3      0  {u'y': 3, u'x': 1, u'index': 0}
1    1.0  4  2      1  {u'y': 2, u'x': 4, u'index': 1}
2    2.0  5  5      2  {u'y': 5, u'x': 5, u'index': 2}

df = df[['label','dict']]
df['label'] = df['label'].apply(str) #Converting integer column 'label' to string
df = df.groupby('label')['dict'].apply(list) 
desired_dict = df.to_dict()
desired_dict 
    {'1.0': [{'index': 0, 'x': 1, 'y': 3}, {'index': 1, 'x': 4, 'y': 2}],
     '2.0': [{'index': 2, 'x': 5, 'y': 5}]}

Answer 2

You can use collections.defaultdict with to_dict :

from collections import defaultdict

# add 'index' series
df = df.reset_index()

# initialise defaultdict
dd = defaultdict(list)

# iterate and append
for d in df.to_dict('records'):
    dd[d['label']].append(d)

Result:

print(dd)

defaultdict(list,
            {1.0: [{'index': 0.0, 'x': 1.0, 'y': 3.0, 'label': 1.0},
                   {'index': 1.0, 'x': 4.0, 'y': 2.0, 'label': 1.0}],
             2.0: [{'index': 2.0, 'x': 5.0, 'y': 5.0, 'label': 2.0}]})

In general, there's no need to convert back to a regular dict , since defaultdict is a subclass of dict .

Answer 3

You can use itertuples and defulatdict :

itertuples returns named tuples to iterate over dataframe:

for row in df.itertuples():
    print(row)
Pandas(Index=0, x=1, y=3, label=1.0)
Pandas(Index=1, x=4, y=2, label=1.0)
Pandas(Index=2, x=5, y=5, label=2.0)

So taking advantage of this:

from collections import defaultdict
dictionary = defaultdict(list)
for row in df.itertuples():
    dummy['x'] = row.x
    dummy['y'] = row.y
    dummy['index'] = row.Index
    dictionary[row.label].append(dummy)

dict(dictionary)
> {1.0: [{'x': 1, 'y': 3, 'index': 0}, {'x': 4, 'y': 2, 'index': 1}],
 2.0: [{'x': 5, 'y': 5, 'index': 2}]}

Answer 4

The quickest solution for what you want is almost along what @cph_sto offers,

>>> df.reset_index().to_dict('records')
[{'index': 0.0, 'label': 1.0, 'x': 1.0, 'y': 3.0}, {'index': 1.0, 'label': 1.0, 'x': 4.0, 'y': 2.0}, {'index': 2.0, 'label': 2.0, 'x': 5.0, 'y': 5.0}]

That is, convert the index to a regular column, then apply the records version of to_dict . Another option of interest:

>>> df.to_dict('index')
{0: {'label': 1.0, 'x': 1.0, 'y': 3.0}, 1: {'label': 1.0, 'x': 4.0, 'y': 2.0}, 2: {'label': 2.0, 'x': 5.0, 'y': 5.0}}

Check the help on to_dict for more.