带有lambda函数的pandas groupby中无法使用.size（）.div（）方法

Question

I'm using the following lines of code to compute the conditional probabilities

    variable = 'variable_name'
    probs = df.groupby(variable).size().div(len(df))
    cond_probs = df.groupby([variable, 'has_income']).size().div(len(df)).div(probs, axis=0, level=variable)

Those results in the following output:

    varibale_name         has_income
    (0.999, 2.0]          False          0.756323
                          True           0.243677
    (2.0, 3.0]            False          0.798372
                          True           0.201628
    (3.0, 16.0]           False          0.809635
                          True           0.190365

I would like to add an additional column to the output as the size of the sample for each group, but I'm not able to rewrite the formula inside the lambda function because the group object doesn't have the same methods as the objects returned by df.groupby() . Example:

    cond_probs =df.groupby([variable, 'has_income']).apply(lambda x: 
    pd.Series({
        'probs': x.size().div(len(df)).div(probs, axis=0, level=variable),
        'size': x.size()
    }))

Error: TypeError: 'numpy.int32' object is not callable

Are there any alternative to achieve these results in a fancy way, without computing two groupby and joining the data frames at the end?

Answer 1

When you use apply with groupby , you don't get a group object, but a slice of the dataframe that corresponds to the relevant group. So x in your case is a DataFrame, not a GroupBy object - treat it the same you'd treat df .

cond_probs = df.groupby([variable, 'has_income']).apply(lambda x: 
  pd.Series({
    'probs': (len(x) / len(df)) / probs[x.iloc[0][variable]],
    'size': len(x)
  })
)

NB if you use .size on a dataframe, it will return the total number of cells - so it's not the same as GroupBy.size ( docs )