Pandas groupby-按列值扩展均值

Question

I'm new to Pandas and somewhat lost on what to do here. I have a dataframe imported from a csv, which (heavily simplified) look like this:

date = ['2013-08-10','2013-08-10','2013-08-10','2013-08-10','2013-08-10',
        '2013-08-10','2013-08-10','2013-08-10','2013-08-10','2013-08-10']
event = ['213','213','213','213','214','214','214','215','215','215']
side = ['A','B','B','B','A','B','A','B','A','B',]
value = [0.193,0.193,0.092,0.027,0.027,0.058,0.027,0.079,0.193,0.159]

df = pd.DataFrame(zip(event,date,side,value),
                  columns=['event','date','side','value'])

  event        date side  value
0   213  2013-08-10    A  0.193
1   213  2013-08-10    B  0.193
2   213  2013-08-10    B  0.092
3   213  2013-08-10    B  0.027
4   214  2013-08-10    A  0.027
5   214  2013-08-10    B  0.058
6   214  2013-08-10    A  0.027
7   215  2013-08-10    B  0.079
8   215  2013-08-10    A  0.193
9   215  2013-08-10    B  0.159

What I want is to sum the values corresponding to each side for every event. This I have achieved with groupby:

groupby = df.groupby(['event','side']).sum()

            value
event side       
213   A     0.193
      B     0.312
214   A     0.054
      B     0.058
215   A     0.193
      B     0.238

But I also want to add a new column with the expanding mean for each side, like this:

            value
event side          roll_mean
213   A     0.193   0
      B     0.312   0
214   A     0.054   0.193
      B     0.058   0.312
215   A     0.193   0.124
      B     0.238   0.185

Note that every event has two side, but it's not always A and B. What I want is something like excel's mean.if function, which computes the expanding mean for all values of the current side, applied to all previous rows. Any help on this would be appreciated.

Answer 1

I think you're actually looking for an expanding mean, not a rolling mean. An expanding mean considers every previous value. I'll start where you left off:

In [63]: res = df.groupby(['event','side']).sum()
In [64]: res
Out[64]: 
            value
event side       
213   A     0.193
      B     0.312
214   A     0.054
      B     0.058
215   A     0.193
      B     0.238

Now we want to groupby side and take the expanding mean:

In [65]: res['expanding_mean'] = res.groupby(level='side').apply(pd.expanding_mean).shift(2)
In [66]: res
Out[66]: 
            value  expanding_mean
event side                       
213   A     0.193             NaN
      B     0.312             NaN
214   A     0.054          0.1930
      B     0.058          0.3120
215   A     0.193          0.1235
      B     0.238          0.1850

Your result needs to be shift ed by 2 since you want the mean to include all previous ones, and not the current one (make sure this is what you actually want, this seems a bit funny). You can replace the shift(2) with len(res.index.levels[1]) to make it a bit more general in case you have more than 2 sides.

Answer 2

I added more 'sides' to your dataframe, so it works when the results aren't just 'A' or 'B'. Is this what you want?

import pandas as pd
import numpy as np
date = ['2013-08-10','2013-08-10','2013-08-10','2013-08-10','2013-08-10',
        '2013-08-10','2013-08-10','2013-08-10','2013-08-10','2013-08-10']
event = ['213','213','213','213','214','214','214','215','215','215']
side = ['A','B','A','B','C','A','C','A','C','A',]
value = [0.193,0.193,0.092,0.027,0.027,0.058,0.027,0.079,0.193,0.159]

df = pd.DataFrame(list(zip(event,date,side,value)),
                columns=['event','date','side','value'])
print(df)

event        date side  value
0   213  2013-08-10    A  0.193
1   213  2013-08-10    B  0.193
2   213  2013-08-10    A  0.092
3   213  2013-08-10    B  0.027
4   214  2013-08-10    C  0.027
5   214  2013-08-10    A  0.058
6   214  2013-08-10    C  0.027
7   215  2013-08-10    A  0.079
8   215  2013-08-10    C  0.193
9   215  2013-08-10    A  0.159


ds = df.groupby(['event','side']).sum()
print(ds)

        value
event side       
213   A     0.285
      B     0.220
214   A     0.058
      C     0.054
215   A     0.238
      C     0.193

ds.reset_index(inplace=True)
ds['exp_mean'] = np.NaN
for s in ds.side.unique():
    ndx = ds[ds.side==s].index
    ds.ix[ndx,'exp_mean'] = pd.expanding_mean(ds.ix[ndx,'value']).shift(1)
ds.set_index(['event', 'side'], inplace=True, drop=True)
print(ds)

            value  exp_mean
event side                 
213   A     0.285       NaN
      B     0.220       NaN
214   A     0.058    0.2850
      C     0.054       NaN
215   A     0.238    0.1715
      C     0.193    0.0540

Answer 3

See this pandas commit (lines 60-78): https://github.com/pandas-dev/pandas/commit/699424027fb657192541bcd0c3d9f9b7d26f2300

`You can now use ``.rolling(..)`` and ``.expanding(..)`` as methods on groupbys. 
These return another deferred object (similar to what ``.rolling()`` and 
``.expanding()`` do on ungrouped pandas objects). You can then operate
 on these ``RollingGroupby`` objects in a similar manner.

Previously you would have to do this to get a rolling window mean per-group:
 .. ipython:: python
    df = pd.DataFrame({'A': [1] * 20 + [2] * 12 + [3] * 8,
                      'B': np.arange(40)})
   df
 .. ipython:: python
    df.groupby('A').apply(lambda x: x.rolling(4).B.mean())
 Now you can do:
 .. ipython:: python
    df.groupby('A').rolling(4).B.mean()`