如何使用 groupby/cut 将 Pandas DataFrame 日期分组到自定义日期范围箱中

Question

I am trying to group dates with a custom range using groupby and cut with no success so far. From the error message being returned, I wonder if cut is trying to process my dates as a number.

I want to group df1['date'] by custom date ranges and then sum the df1['HDD'] values. The custom ranges are found in df2 :

import pandas as pd
df1 = pd.DataFrame( {'date': ['2/1/2015', '3/2/2015', '3/3/2015', '3/4/2015','4/17/2015','5/12/2015'],
                             'HDD' : ['7.5','8','5','23','11','55']})
    HDD  date
0   7.5 2/1/2015
1   8   3/2/2015
2   5   3/3/2015
3   23  3/4/2015
4   11  4/17/2015
5   55  5/12/2015

df2 has the custom date ranges:

df2 = pd.DataFrame( {'Period': ['One','Two','Three','Four'],
                     'Start Dates': ['1/1/2015','2/15/2015','3/14/2015','4/14/2015'],
                     'End Dates' : ['2/14/2015','3/13/2015','4/13/2015','5/10/2015']})

    Period  Start Dates End Dates
0   One     1/1/2015    2/14/2015
1   Two     2/15/2015   3/13/2015
2   Three   3/14/2015   4/13/2015
3   Four    4/14/2015   5/10/2015

My Desired output is to group df1 by the custom date ranges and aggregate the HDD values for each Period. Should output something like this:

   Period    HDD
0  One       7.5
1  Two       36
2  Three     0
3  Four      11

Here is one example of what I have tried to use custom grouping:

df3 = df1.groupby(pd.cut(df1['date'], df2['Start Dates'])).agg({'HDD': sum})

...and here is the error I get:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-103-55ea779bcd73> in <module>()
----> 1 df3 = df1.groupby(pd.cut(df1['date'], df2['Start Dates'])).agg({'HDD': sum})

/opt/conda/lib/python3.5/site-packages/pandas/tools/tile.py in cut(x, bins, right, labels, retbins, precision, include_lowest)
    112     else:
    113         bins = np.asarray(bins)
--> 114         if (np.diff(bins) < 0).any():
    115             raise ValueError('bins must increase monotonically.')
    116 

/opt/conda/lib/python3.5/site-packages/numpy/lib/function_base.py in diff(a, n, axis)
   1576         return diff(a[slice1]-a[slice2], n-1, axis=axis)
   1577     else:
-> 1578         return a[slice1]-a[slice2]
   1579 
   1580 

TypeError: unsupported operand type(s) for -: 'str' and 'str'

• Is cut trying to process my date ranges as numbers?
• Do I need to explicitly convert my dates as datetime objects (tried this but maybe was going about it correctly)?

Thanks for any suggestions offered!

Answer 1

This works if you convert all your dates form dtype string to datetime.

df1['date'] = pd.to_datetime(df1['date'])

df2['End Dates'] = pd.to_datetime(df2['End Dates'])

df2['Start Dates'] = pd.to_datetime(df2['Start Dates'])

df1['HDD'] = df1['HDD'].astype(float)

df1.groupby(pd.cut(df1['date'], df2['Start Dates'])).agg({'HDD': sum})

Output:

                           HDD
date                          
(2015-01-01, 2015-02-15]   7.5
(2015-02-15, 2015-03-14]  36.0
(2015-03-14, 2015-04-14]   NaN

Adding labels:

df1.groupby(pd.cut(df1['date'], df2['Start Dates'], labels=df2.iloc[:-1,1])).agg({'HDD': sum})

Output:

        HDD
date       
One     7.5
Two    36.0
Three   NaN