基于另一个系列的熊猫高效分组

Question

I need to erform a grouped operation which is based on another boolean column in my DataFrame . It is most easily seen in an example: I have the following DataFrame :

    b          id   
0   False      0
1   True       0
2   False      0
3   False      1
4   True       1
5   True       2
6   True       2
7   False      3
8   True       4
9   True       4
10  False      4

and would like to obtain a column, whose elements are True if the b column is True and it is the last time it is True for a given id :

    b          id    lastMention
0   False      0     False
1   True       0     True
2   False      0     False
3   False      1     False
4   True       1     False
5   True       2     True
6   True       3     True
7   False      3     False
8   True       4     False
9   True       4     True
10  False      4     False

I have a code that achieves this, although inefficiently:

def lastMentionFun(df):
    b = df['b']
    a = b.sum()
    if a > 0:
        maxInd = b[b].index.max()
        df.loc[maxInd, 'lastMention'] = True
    return df

df['lastMention'] = False
df = df.groupby('id').apply(lastMentionFun)

Can someone propose what is the correct pythonic approach to do this nice and fast?

Answer 1

You can first filter values where True in column b and then get max index value with groupby and aggregating max :

print (df[df.b].reset_index().groupby('id')['index'].max())
id
0    1
1    4
2    6
4    9
Name: index, dtype: int64

Then replace values False by index values with loc :

df['lastMention'] = False
df.loc[df[df.b].reset_index().groupby('id')['index'].max(), 'lastMention'] = True

print (df)
        b  id  lastMention
0   False   0        False
1    True   0         True
2   False   0        False
3   False   1        False
4    True   1         True
5    True   2        False
6    True   2         True
7   False   3        False
8    True   4        False
9    True   4         True
10  False   4        False

Another solution - get max index values with groupby and apply , then test membership of values in index with isin - output is boolean Series :

print (df[df.b].groupby('id').apply(lambda x: x.index.max()))
id
0    1
1    4
2    6
4    9
dtype: int64

df['lastMention'] = df.index.isin(df[df.b].groupby('id').apply(lambda x: x.index.max()))
print (df)
        b  id lastMention
0   False   0       False
1    True   0        True
2   False   0       False
3   False   1       False
4    True   1        True
5    True   2       False
6    True   2        True
7   False   3       False
8    True   4       False
9    True   4        True
10  False   4       False

Answer 2

Not sure if this is the most efficient method, but it uses only built-in functions (the main one being "cumsum" and then max to check that it equals the last one - pd.merge is just used to put the max back in the table, maybe there's a better way for doing that?).

df['cum_b']=df.groupby('id', as_index=False).cumsum()
df = pd.merge(df, df[['id','cum_b']].groupby('id', as_index=False).max(), how='left', on='id', suffixes=('','_max'))
df['lastMention'] = np.logical_and(df.b, df.cum_b == df.cum_b_max)

PS The dataframe you specified in the example changes slightly from the first to the second snippet, I hope I've interpreted your request correctly!