Python - how to pass a result from group by to Pivot?

Question

My goal was to apply pivot function to a data frame that contains duplicate records. I solved it by adding a unique column to the data frame:

my_df['id_column'] = range(1, len(my_df.index)+1)

df_pivot = my_df.pivot(index ='id_column', columns = 'type', values = 'age_16_18').fillna(0).astype(int)

I want to figure out how to apply pivot to the data frame without deleting duplicates or using pivot_table? By fist grouping by multiple columns, and then passing the result to the pivot function. I'm not sure how to pass a result after grouping to pivot.

    year  category  state_name  type    is_state gender age_16_18 age_18_30
0   2001  Foreigners  CA       Convicts   0       M       8          5
1   2001  Indians     NY       Convicts   0       F       5          2 
2   2005  Foreigners  NY       Others     1       M       0          9
3   2009  Indians     NJ       Detenus    0       F       7          0

Answer 1

It's not entirely clear what you're attempting but see if you can get some inspiration from the following approaches. What columns are you wishing to group by?

import pandas
my_df = pandas.DataFrame( { 'year' : [2001, 2001, 2005, 2009] ,
                            'category' : ['Foreigners','Indians','Foreigners','Indians'] ,
                            'state_name': ['CA','NY','NY','NJ' ],
                            'type': ['Convicts', 'Convicts','Others','Detenus'],
                            'is_state' : [0,0,1,0] ,
                            'gender' : ['M','F','M','F'],
                            'age_16_18':[8,5,0,7],
                            'age_18_30' : [5,2,9,0] }, columns=[ 'year','category','state_name','type','is_state','gender','age_16_18','age_18_30'])

>>> my_df.pivot( columns = 'type', values = 'age_16_18' )
type  Convicts  Detenus  Others
0          8.0      NaN     NaN
1          5.0      NaN     NaN
2          NaN      NaN     0.0
3          NaN      7.0     NaN

>>> my_df['key'] = my_df.category.str.cat(my_df.gender)

>>> my_df.pivot( index='key', columns = 'type', values = 'age_16_18' )
type         Convicts  Detenus  Others
key
ForeignersM       8.0      NaN     0.0
IndiansF          5.0      7.0     NaN