Python、Pandas：使用 GroupBy.groups 描述将其应用于另一个分组

Question

Let's consider a DataFrame that contains 1 row of 2 values per each day of the month of Jan 2010:

date_range = pd.date_range(dt(2010,1,1), dt(2010,1,31), freq='1D')
df = pd.DataFrame(data = np.random.rand(len(date_range),2), index = date_range)

I split that DataFrame into a list of 5 DataFrames, each of them containing 1 week worth of data from the original: df_weeks = [g for n, g in df.groupby(pd.TimeGrouper('W'))]

If I type df.groupby(pd.TimeGrouper('W')).groups I can see a dict explaining how the groups are split:

{Timestamp('2010-01-03 00:00:00', freq='W-SUN'): 3,
 Timestamp('2010-01-10 00:00:00', freq='W-SUN'): 10,
 Timestamp('2010-01-17 00:00:00', freq='W-SUN'): 17,
 Timestamp('2010-01-24 00:00:00', freq='W-SUN'): 24,
 Timestamp('2010-01-31 00:00:00', freq='W-SUN'): 31}

Let's consider another TimeSerie that has sparser data yet overlaps with the first one:

observations = pd.DataFrame(data =np.random.rand(5,2), index = (dt(2010,1,12), dt(2010,1,18), dt(2010,1,20), dt(2010,1,22), dt(2010,1,28)))

If I run the same code obs_weeks = [g for n, g in observations.groupby(pd.TimeGrouper('W'))] , obviously it will returns less DataFrames in the list, as the data covers less span. observations.groupby(pd.TimeGrouper('W')).groups returns :

{Timestamp('2010-01-17 00:00:00', freq='W-SUN'): 1,
 Timestamp('2010-01-24 00:00:00', freq='W-SUN'): 4,
 Timestamp('2010-01-31 00:00:00', freq='W-SUN'): 5}

But would there be a way to reuse the groups of the first DataFrame.GroupBy to apply it to the second one? ie that would mean in that specific case ending up with a variable obs_weeks containing 5 DataFrames spanning the same time range as df_weeks , 2 of them being empty ?

Answer 1

One simple solution to your problem would be to make sure the observations dataframe contains all the dates that the df dataframe does. You can do this with the reindex method. You will then have the exact same groups. You can also use resample('W') instead of groupby(pd.Timegrouper('W'))

obs2 = observations.reindex(df.index)

obs2.resample('W').groups

{Timestamp('2010-01-03 00:00:00', freq='W-SUN'): 3,
 Timestamp('2010-01-10 00:00:00', freq='W-SUN'): 10,
 Timestamp('2010-01-17 00:00:00', freq='W-SUN'): 17,
 Timestamp('2010-01-24 00:00:00', freq='W-SUN'): 24,
 Timestamp('2010-01-31 00:00:00', freq='W-SUN'): 31}

And if we do a simple aggregation like sum we can see the results of both frames

df.resample('W').sum()

                 0         1
2010-01-03  1.990558  2.555191
2010-01-10  2.707777  3.771756
2010-01-17  2.799897  3.353363
2010-01-24  3.165479  2.778870
2010-01-31  4.946577  3.394211

And now with obs2 which has 2 missing groups

obs2.resample('W').sum()

                   0         1
2010-01-03       NaN       NaN
2010-01-10       NaN       NaN
2010-01-17  0.172341  0.137136
2010-01-24  1.752472  2.375306
2010-01-31  0.711525  0.124271