Pandas与groupby的部分元素的累积和

Question

Apologies if this question has already been asked, but thank you in advance for your help.

In this "unpivoted" dataset, there are Orders composed of several Lots . Each Lot has a given Point value, as below:

CustID     Date         OrderNum   LotNum   PtsPerLot
A123       1/1/2015     1234       A        2            
A123       1/1/2015     1234       B        10
A123       1/1/2015     5678       A        7

My objective is to create a CUMULATIVE_POINTS_PER_YEAR column representing the cumulative sum of POINTS_PER_ORDER , which is itself a sum of PtsPerLot , at each Lot level. So, for a given lot, CumPtsPerYear would show the cumulative total of all POINTS_PER_ORDER for an account in a given year.

CustID     Date         OrderNum   LotNum   PtsPerLot    *PtsPerOrder*    *CumPtsPerYear*
A123       1/1/2015     1234       A        2            12              12
A123       1/1/2015     1234       B        10           12              12
A123       1/1/2015     5678       A        7            7               19

Any ideas? I've tried groupby.cumsum on PtsPerLot and another groupby.cumsum on PtsPerOrder , but it isn't producing what I need.

Answer 1

First, calculate PtsPerOrder . Use transform to broadcast along the actual index of your dataframe the result of the calculation in each group:

df['PtsPerOrder'] = df.groupby('OrderNum')['PtsPerLot'].transform(sum)

Then take the first element of that new column in each group:

df['CumPtsPerYear'] = df.groupby('OrderNum')['PtsPerOrder'].head(1)

df
Out[27]: 
  CustID      Date  OrderNum LotNum  PtsPerLot  PtsPerOrder  CumPtsPerYear
0   A123  1/1/2015      1234      A          2           12           12.0
1   A123  1/1/2015      1234      B         10           12            NaN
2   A123  1/1/2015      5678      A          7            7            7.0

End the calculation by doing the cumulative sum you are searching for. It will skip the NA values. You complete your dataframe with a forward fill:

df['CumPtsPerYear'].cumsum().ffill()

0    12.0
1    12.0
2    19.0

Answer 2

First you need to use a transformation :

df['*PtsPerOrder*'] = df.groupby('OrderNum')['PtsPerLot'].transform(sum)

Then in order to create the other one, I didn't find another way that to find the max of each group, do a cumsum on that, and merge that back in:

weird_cumsum = df.groupby('OrderNum')['*PtsPerOrder*'].max().cumsum().to_frame()
weird_cumsum.columns = ['*CumPtsPerYear*']
weird_cumsum

          *CumPtsPerYear*
OrderNum                 
1234                   12
5678                   19

df.merge(weird_cumsum, left_on='OrderNum', right_index=True, how='left')

Result is as expected:

  CustID       Date  OrderNum LotNum  PtsPerLot  *PtsPerOrder*  *CumPtsPerYear* 
0   A123 2015-01-01      1234      A          2             12             12  
1   A123 2015-01-01      1234      B         10             12             12   
2   A123 2015-01-01      5678      A          7              7             19   

Answer 3

To get to the first part of your question, PtsPerOrder , you need a transformation . sum is an aggregation. So use .transform :

In [10]: df
Out[10]:
            Date  OrderNum LotNum  PtsPerLot
CustID
A123    1/1/2015      1234      A          2
A123    1/1/2015      1234      B         10
A123    1/1/2015      5678      A          7

In [11]: df.groupby('OrderNum')['PtsPerLot'].transform('sum')
Out[11]:
CustID
A123    12
A123    12
A123     7
dtype: int64

And use that to create a new column...

In [13]: df['PtsPerOrder'] = df.groupby('OrderNum')['PtsPerLot'].transform('sum')

In [14]: df
Out[14]:
            Date  OrderNum LotNum  PtsPerLot  PtsPerOrder
CustID
A123    1/1/2015      1234      A          2           12
A123    1/1/2015      1234      B         10           12
A123    1/1/2015      5678      A          7            7

I'm still not grokking your specification for CumPtsPerYear ...