使用 pandas 变换的多个函数

Question

I have a dataset that looks like this:

   entity_id transaction_date transaction_month  net_flow    inflow   outflow
0         51       2018-07-02        2018-07-01  10161.06  20161.06  10000.00
1         51       2018-07-03        2018-07-01   5823.73   5867.37     43.64
2         51       2018-07-05        2018-07-01  17835.79  24107.29   6271.50
3         51       2018-07-06        2018-07-01  -3544.72  31782.84  35327.56
4         51       2018-07-09        2018-07-01  18252.42  18332.42     80.00

I am trying to calculate the rolling metrics across the entity_id field using rolling and transform . I have multiple variables I'd like to create and would prefer to run them in a single call.

For example, if I were to create these measures using agg , I would execute something like this:

transactions = (
    raw_transactions
    .groupby(['entity_id','transaction_month'])[['inflow','outflow']]
    .agg([
        'sum','skew',
        ( 'coef_var', lambda x: x.std() / x.mean() ),
        ( 'kurtosis', lambda x: x.kurtosis() )
        ])
    .reset_index()
)

However, I'm unable to reproduce this using transform . When I try to pass functions using either a dict or list, I get a TypeError due to list or dict being unhashable.

>>> transactions.groupby(['entity_id'])[['inflow','outflow']].transform(['skew','mean'])

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-62-4ef49d836b3f> in <module>
----> 1 transactions.groupby(['entity_id'])[['inflow','outflow']].transform(['skew','mean'])

/jupyter/packages/pandas/core/groupby/generic.py in transform(self, func, engine, engine_kwargs, *args, **kwargs)
   1354 
   1355         # optimized transforms
-> 1356         func = self._get_cython_func(func) or func
   1357 
   1358         if not isinstance(func, str):

/jupyter/packages/pandas/core/base.py in _get_cython_func(self, arg)
    335         if we define an internal function for this argument, return it
    336         """
--> 337         return self._cython_table.get(arg)
    338 
    339     def _is_builtin_func(self, arg):

TypeError: unhashable type: 'list'

Answer 1

I don't think it is possible with transform . You have two workarounds (at least). Either merge the result of groupby.agg on the original dataframe:

tmp_ = (
    raw_transactions
    .groupby(['entity_id','transaction_month'])[['inflow','outflow']]
    .agg([
        'sum','skew',
        ( 'coef_var', lambda x: x.std() / x.mean() ),
        ( 'kurtosis', lambda x: x.kurtosis() )
        ]) #no reset_index here
)
# need to flatten multiindex columns
tmp_.columns = ['_'.join(cols) for cols in tmp_.columns] 

# then merge with original dataframe
res = raw_transactions.merge(tmp_, on=['entity_id','transaction_month'])

or use a list comprehension over the different function to transform in a concat with the original data

# group once
gr = raw_transactions.groupby(['entity_id'])[['inflow','outflow']]

#concat each dataframe of transformed function with otiginal data
res = pd.concat([raw_transactions] + 
                [gr.transform(func) 
                 for func in ('skew', 'mean', lambda x: x.std() / x.mean() )], 
                axis=1, keys=('', 'skew', 'mean', 'coef_var'))

then you can work on columns name