了解 pandas groupby().agg() 值

Question

I cam across some code for testing Simpson's Paradox, and I'm confused about how it works.

The data is in this form:

and when I run

gb = df_.groupby(["kidney_stone_size", "treatment"]).agg([np.sum, lambda x: len(x)])
gb

I get

I can't fully understand what df_.groupby(["kidney_stone_size", "treatment"]).agg([np.sum, lambda x: len(x)]) does.

For one, the aggragate data appears to be calculated from the omitted columns in the groupby part, as when I do

gb = df_.groupby(["recovery", "treatment"]).agg([np.sum, lambda x: len(x)])

I get

So is that the default behaviour - that the aggregate data is calculated for the missing columns?

I know you can specify columns explicitly in a dictionary, but I'm trying to understand the code as is.

What exactly is being calculated by the .agg([np.sum, lambda x: len(x)]) ?

ie what exactly is np.sum being applied to, and likewise lambda x: len(x) ?

Please understand that there may be some conceptual gaps in my understanding that might make what is obvious from the outside non-obvious to me. Any help much appreciated.

Answer 1

So is that the default behaviour - that the aggregate data is calculated for the missing columns?

I think yes, if not specify column for processing after groupby pandas use all columns not used in groupby and apply aggregate functions.

What exactly is being calculated by the.agg([np.sum, lambda x: len(x)])

Here sum for non numeric columns working like join , for numeric get sum , your custom function lambda x: len(x) return length of groups - numeric and non numeric.

df_ = pd.DataFrame({
        'kidney_stone_size':list('aaaaaa'),
         'recovery':[4,5,4,5,5,4],
         'col1':[1,3,5,7,1,0],
         'col2':['new'] * 6,
         'treatment':list('aaabbb')
})

df = df_.groupby(["kidney_stone_size", "treatment"]).agg([np.sum, lambda x: len(x)])
print (df)
                            recovery            col1                  col2  \
                                 sum <lambda_0>  sum <lambda_0>        sum   
kidney_stone_size treatment                                                  
a                 a               13          3    9          3  newnewnew   
                  b               14          3    8          3  newnewnew   

                                        
                            <lambda_0>  
kidney_stone_size treatment             
a                 a                  3  
                  b                  3  

But if use only aggregate function working with numeric like sum , pandas by default omit non numeric columns:

df = df_.groupby(["kidney_stone_size", "treatment"]).sum()
print (df)
                             recovery  col1
kidney_stone_size treatment                
a                 a                13     9
                  b                14     8