熊猫在groupby中按条件过滤

Question

I have a matrix/dataframe with times and values:

     # time             # Value
M = [[2018-08-08 12:00:00, 5],
     [2018-08-08 12:00:00, 7],
     [2018-08-08 13:00:00, 2],]

I want to group by hour, then calculate the mean value of the group, and then modify/reduce each group so that it has only values <= this mean.

Current version:

grouped = M.groupby(pd.Grouper(key='time', freq='1h'))
means = grouped['value'].mean().values # np.array([6, 2])

Here I'm stuck. I get the mean values for each group. But I don't know how to reduce the "grouped" so that the condition applies that grouped[grouped['value'] <= mean] for that group.

Appreciate any suggestions.

---

Expected output:

N = [[2018-08-08 12:00:00, 5], # as 5 <= 6 where 6 is the mean of the first group
     [2018-08-08 13:00:00, 2]] # as 2 is <= 2 where 2 is the mean of the second group

Answer 1

Use GroupBy.transform for Series with same size as original DataFrame filled by aggregated values, so boolean indexing working very nice:

M = [['2018-08-08 12:00:00', 5],
     ['2018-08-08 12:00:00', 7],
     ['2018-08-08 13:00:00', 2]]

M = pd.DataFrame(M, columns=['time','value'])
M['time'] = pd.to_datetime(M['time'])
print (M)
                 time  value
0 2018-08-08 12:00:00      5
1 2018-08-08 12:00:00      7
2 2018-08-08 13:00:00      2

s = M.groupby(pd.Grouper(key='time', freq='1h'))['value'].transform('mean')
print (s)
0    6
1    6
2    2
Name: value, dtype: int64

mean = 5
df = M[s <= mean]
print (df)
                 time  value
2 2018-08-08 13:00:00      2

EDIT:

You can also compare by columns values:

df1 = M[M['value'] <= s]
print (df1)
                 time  value
0 2018-08-08 12:00:00      5
2 2018-08-08 13:00:00      2