Apply Function or Lambda to Pandas GROUPBY

Question

I would like to apply a specific function (in this case a logit model) to a dataframe which can be grouped (by the variable "model"). I know the task can be performed through a loop, however I believe this to be inefficient at best. Example code below:

import pandas as pd
import numpy as np
import statsmodels.api as sm
df1=pd.DataFrame(np.random.randint(0,100,size=(100,10)),columns=list('abcdefghij'))
df2=pd.DataFrame(np.random.randint(0,100,size=(100,10)),columns=list('abcdefghij'))
df1['model']=1
df1['target']=np.random.randint(2,size=100)
df2['model']=2
df2['target']=np.random.randint(2,size=100)
data=pd.concat([df1,df2])
### Clunky, but works...  
for i in range(1,2+1):
    lm=sm.Logit(data[data['model']==i]['target'],
                sm.add_constant(data[data['model']==i].drop(['target'],axis=1))).fit(disp=0)
    print(lm.summary2())
### Can this work?  
def elegant(self):
    lm=sm.Logit(data['target'],
                sm.add_constant(data.drop(['target'],axis=1))).fit(disp=0)
better=data.groupby(['model']).apply(elegant)

If the above groupby can work, is this a more efficient way to perform than looping?

Answer 1

This could work:

def elegant(df):
lm = sm.Logit(df['target'],
              sm.add_constant(df.drop(['target'],axis=1))).fit(disp=0)
return lm 

better = data.groupby('model').apply(elegant)

Using .apply you passe the dataframe groups to the function elegant so elegant has to take a dataframe as the first argument here. Also your function needs to return the result of your calculation lm .

For more complexe functions the following structure can be used:

def some_fun(df, kw_param=1):
# some calculations to df using kw_param
return df

better = data.groupby('model').apply(lambda group: some_func(group, kw_param=99))