Dictionary from DataFrame: key from one column, values from multiple rows and columns excluding NaN

Question

I want to create dictionary from pd.DataFrame where I want id to be key and all value_x are values but excluding NaN

Dataframe newdf :

     id    name  value_1  value_2  value_3
0    ant   jay   10.2     3.5      4.7
1    ant   ann   5.7      10.2     NaN
2    bee   will  7.4      NaN      NaN
3    bee   dave  12.4     1.3      6.9
4    bee   ed    0.8      NaN      NaN
5    cat   kit   NaN      NaN      5.2

The expected outcome (value is sorted row by row) is

{ant:(10.2,3.5,4.7,5.7,10.2), bee:(7.4,12.4,1.3,6.9,0.8), cat:(5.2)}

I am trying to use .to_dict() but it does work yet

newdf.groupby('id').apply(newdf.iloc[:,-3:].to_dict())

or

dict(zip(newdf.id, newdf.iloc[:,-3:]))

Answer 1

Use:

d = df.set_index('id').iloc[:, -3:].stack().groupby(level=0).apply(tuple).to_dict()
print (d)
{'bee': (7.4, 12.4, 1.3, 6.9, 0.8), 'cat': (5.2,), 'ant': (10.2, 3.5, 4.7, 5.7, 10.2)}

Detail:

print (df.set_index('id').iloc[:, -3:].stack())

id          
ant  value_1    10.2
     value_2     3.5
     value_3     4.7
     value_1     5.7
     value_2    10.2
bee  value_1     7.4
     value_1    12.4
     value_2     1.3
     value_3     6.9
     value_1     0.8
cat  value_3     5.2
dtype: float64

If ordering is necesary and use pandas 0.21.0 is possible generate OrderedDict :

from collections import OrderedDict

d = (df.set_index('id')
       .iloc[:, -3:]
       .stack()
       .groupby(level=0)
       .apply(tuple)
       .to_dict(into=OrderedDict))
print (d)

OrderedDict([('ant', (10.2, 3.5, 4.7, 5.7, 10.2)), 
             ('bee', (7.4, 12.4, 1.3, 6.9, 0.8)), 
             ('cat', (5.2,))])