pandas DataFrame以每组为基础插入/重新采样每日数据

Question

I've got a dataframe that looks like this:

userid      date          count
a           2016-12-01    4
a           2016-12-03    5
a           2016-12-05    1
b           2016-11-17    14
b           2016-11-18    15
b           2016-11-23    4

The first column is a user id, the second column is a date (resulting from a groupby(pd.TimeGrouper('d')), and the third column is a daily count. However, per user, I would like to ensure that any days missing between a user's min and max date are filled in to be 0 on a per user basis. So if I am starting with a data frame like the above, I end up with a data frame like this:

   userid      date          count
    a           2016-12-01    4
    a           2016-12-02    0
    a           2016-12-03    5
    a           2016-12-04    0
    a           2016-12-05    1
    b           2016-11-17    14
    b           2016-11-18    15
    b           2016-11-19    0
    b           2016-11-20    0
    b           2016-11-21    0
    b           2016-11-22    0
    b           2016-11-23    4

I know that there are various methods available with a pandas data frame to resample (with options to pick to interpolate forwards, backwards, or by averaging) but how would I do this in the sense above, where I want a continuous time series for each userid but where the dates of the time series are different per user?

Here's what I tried that hasn't worked:

grouped_users = user_daily_counts.groupby('user').set_index('timestamp').resample('d', fill_method = None)

However this throws an error AttributeError: Cannot access callable attribute 'set_index' of 'DataFrameGroupBy' objects, try using the 'apply' method . I'm not sure how I'd be able to use the apply method while bringing forward all columns as I'd like to do.

Thanks for any suggestions!

Answer 1

You can use groupby with resample , but first need Datetimeindex created by set_index .
( need pandas 0.18.1 and higher )

Then fill NaN by 0 by asfreq with fillna .

Last remove column userid and reset_index :

df = df.set_index('date')
       .groupby('userid')
       .resample('D')
       .asfreq()
       .fillna(0)
       .drop('userid', axis=1)
       .reset_index()

print (df)
   userid       date  count
0       a 2016-12-01    4.0
1       a 2016-12-02    0.0
2       a 2016-12-03    5.0
3       a 2016-12-04    0.0
4       a 2016-12-05    1.0
5       b 2016-11-17   14.0
6       b 2016-11-18   15.0
7       b 2016-11-19    0.0
8       b 2016-11-20    0.0
9       b 2016-11-21    0.0
10      b 2016-11-22    0.0
11      b 2016-11-23    4.0

If want dtype of column count integer add astype :

df = df.set_index('date') \
       .groupby('userid') \
       .resample('D') \
       .asfreq() \
       .fillna(0) \
       .drop('userid', axis=1) \
       .astype(int) \
       .reset_index()

print (df)
   userid       date  count
0       a 2016-12-01      4
1       a 2016-12-02      0
2       a 2016-12-03      5
3       a 2016-12-04      0
4       a 2016-12-05      1
5       b 2016-11-17     14
6       b 2016-11-18     15
7       b 2016-11-19      0
8       b 2016-11-20      0
9       b 2016-11-21      0
10      b 2016-11-22      0
11      b 2016-11-23      4