如何过滤 DataFrame 以在 Pandas 的列中的特定单词列表之后保留行？

Question

How can I filter a dataframe that keeps rows after specific list of words that is sorted by date? I have a df that looks like

    Name    Date    Event   Col1
0   Sam 1/1/2020    Apple   Test1
1   Sam 1/2/2020    Apple   Test2
2   Sam 1/3/2020    BALL    Test1
3   Sam 1/3/2020    CAT Test2
4   Sam 1/5/2020    BALL    Test2
5   Sam 1/6/2020    Apple   Test3
6   Nick    1/5/2020    CAT Test3
7   Nick    1/6/2020    BALL    Test3
8   Nick    1/7/2020    Apple   Test3
9   Nick    1/8/2020    Apple   Test4
10  Cat 1/1/2020    Apple   Test1
11  Cat 1/2/2020    Bat Test2




 df=pd.DataFrame({'Name': {0: 'Sam',
  1: 'Sam',
  2: 'Sam',
  3: 'Sam',
  4: 'Sam',
  5: 'Sam',
  6: 'Nick',
  7: 'Nick',
  8: 'Nick',
  9: 'Nick',
  10: 'Cat',
  11: 'Cat '},
 'Date': {0: '1/1/2020',
  1: '1/2/2020',
  2: '1/3/2020',
  3: '1/3/2020',
  4: '1/5/2020',
  5: '1/6/2020',
  6: '1/5/2020',
  7: '1/6/2020',
  8: '1/7/2020',
  9: '1/8/2020',
  10: '1/1/2020',
  11: '1/2/2020'},
 'Event': {0: 'Apple',
  1: 'Apple',
  2: 'BALL',
  3: 'CAT',
  4: 'BALL',
  5: 'Apple',
  6: 'CAT',
  7: 'BALL',
  8: 'Apple',
  9: 'Apple',
  10: 'Apple',
  11: 'Bat'},
 'Col1': {0: 'Test1',
  1: 'Test2',
  2: 'Test1',
  3: 'Test2',
  4: 'Test2',
  5: 'Test3',
  6: 'Test3',
  7: 'Test3',
  8: 'Test3',
  9: 'Test4',
  10: 'Test1',
  11: 'Test2'}})

I would like to keep the rows after earliest date where BALL or CAT occurs in my event. So in my example, I would need to eliminate 1st 2 rows and 11th row since we have Apple as the first events.

I tried using

event_filter = ['BALL','CAT']
df = df.loc[df['Event'].isin(event_filter)]

I also tried to remove the subset based on events but it removed 8th row as well.

Any help would be appreciated. The result I am expecting is:

    Name    Date    Event   Col1
0   Sam 1/3/2020    BALL    Test1
1   Sam 1/3/2020    CAT Test2
2   Sam 1/5/2020    BALL    Test2
3   Sam 1/6/2020    Apple   Test3
4   Nick    1/5/2020    CAT Test3
5   Nick    1/6/2020    BALL    Test3
6   Nick    1/7/2020    Apple   Test3
7   Nick    1/8/2020    Apple   Test4
8   Cat 1/2/2020    Bat Test2

Answer 1

It was a little hard to follow (did you switch the event filter from Bat to BALL? :D ), and it seems like you are trying the get the first event per person?

If so I think you need to split the dataframe by name, filter as needed and then recombine.

here's small function to get the first occurence:

def get_min_index(ser, event_filter):

    in_event = ser.isin(event_filter)
    return in_event.loc[in_event].index[0]

Then assuming your df is already sorted as you want it.

tdf_lst = []
names = df['Name'].unique()

for name in names:

    tdf = df.loc[df['Name']==name, :] # filter for the individual name
    min_idx = get_min_index(tdf['Event'], event_filter) # get the first index
    tdf = tdf.loc[min_idx:,:] # select from the first index to the last
    tdf_lst.append(tdf)
    
df_fltrd = pd.concat(tdf_lst)

maybe there's a more elegant solution but hopefully that's that you are looking for

Answer 2

how about something like this? Also, it seems there is typo. For last row, there is Bat, was this supposed to be BALL? (According to your expected output )

lst = ['CAT', 'BALL']

check if that selected element of list exists in the event. if exist, give it 1 if doesn't exist, give it 0.

df['C'] = np.where(df['Event'].isin(lst), 1, 0)

after this, we can do cumsum for column C and filter the rows. This can be done by using groupby on Name and doing cumsum on column c and check if there exists cumsum greater than 0. The greater than 0 only happens if there exists those element of list in that event for that groupby (Name)

df = df.loc[df.groupby('Name')['C'].cumsum()>0].reset_index(drop=True)
df.drop('C', 1, inplace=True)
print (df)

   Name      Date  Event   Col1
0   Sam  1/3/2020   BALL  Test1
1   Sam  1/3/2020    CAT  Test2
2   Sam  1/5/2020   BALL  Test2
3   Sam  1/6/2020  Apple  Test3
4  Nick  1/5/2020    CAT  Test3
5  Nick  1/6/2020   BALL  Test3
6  Nick  1/7/2020  Apple  Test3
7  Nick  1/8/2020  Apple  Test4