使用熊猫删除基于行的列的值

Question

I'm new to pandas and need some help with deleting rows fulfilling certain conditions from my original table below.

Table1 (original table):

    ID  SerialNo    calls
1   171723  Blue    2
2   171723  Green   3
3   171723  Blue    4
4   171723  Yellow  5
5   171723  Blue    1
6   171724  Green   1
7   171724  Yellow  2
8   171724  Green   3
9   171724  Green   4
10  171724  Green   5
11  171724  Yellow  6

Table1_mod (filtered table):

    ID  SerialNo    calls
 1  171723  Blue    2
 2  171723  Green   3
 3  171723  Blue    4
 4  171723  Yellow  5
 5  171724  Green   1
 6  171724  Yellow  2
 7  171724  Green   3
 8  171724  Green   4
 9  171724  Green   5
10  171724  Yellow  6

I want to obtain the modified table based on the following:

ID column contain unique numbers. For example, for all rows with '171723', and under column 'SerialNO' I'm interested in 'Blue'. So I want the last row of '171723' deleted because Blue (SerialNo column) has a row corresponding to '1' (calls column) which is smaller than than '2' (the first occurrence of Blue on Calls column for '171723')

How can I write a pandas code to remove rows fulfilling these conditions?

Thanks

Answer 1

Just to clarify: you want to look into groups of id numbers, find the first occurrence of 'Blue' in the Serial Numbers column, and remove any other rows in this group where the Serial Number is also Blue and the calls value is less than the calls value in the first 'Blue' row?

I would first group your dataframe by Id

id_groups = Table1.groupby('ID')

Then define a function to do your filtering:

def blue_filter(group):
    blues = group[group['SerialNo'] == 'Blue']
    try:
        first_blue_call =  blues['calls'].iloc[0]
    except:
        return group
    return group[(group['SerialNo'] != 'Blue') | (group['calls'] >= first_blue_call)] 

Now apply that function to your groups and reform the groups as a new dataframe and reset the indices

Table1_mod = id_groups.apply(blue_filter)
Table1_mod.index =  Table1_mod.index.levels[1]

Answer 2

I think you can do this all in one go using apply and a separate dict that keeps track of your maximum value of calls. This also deals with the fact that I think you want to throw out any row where the call number is lower than any previous row for that ID, SerialNo combo.

max_dict = {}

def keep_row(row):
    if row.calls > max_dict.get((row.ID, row.SerialNo), 0):
        max_dict[(row.ID, row.SerialNo)] = row.calls
        return True
    else:
        return False

Table1_mod = Table1[Table1.apply(keep_row, axis=1)]