如何根据 Pandas dataframe 中的某些列仅保留最后一个条目？

Question

I have a dataframe like below with the following columns - TEST_NUM, SITE_NUM, HEAD_NUM, RESULT

Here is sample data in it -

________________________________________ 
TEST_NUM | HEAD_NUM | SITE_NUM | RESULT
________________________________________ 
10000   |  1          | 0       |  P
________________________________________ 
10000   |  1          | 1       |  F          --> Should be retest, as result is F
________________________________________ 
10000   |  1          | 2       |  F          ---> Should be retest, as result is F
________________________________________ 
10000   |  1          | 3       |  P 
________________________________________ 
10000   |  1          | 1       |  P          ----> Retest done, finally Pass
________________________________________ 
10000   |  1          |  2      |  P          ----> Retest done, finally Pass

The above data is data from a testing device that works on 4 sites {0,1,2,3}. As you can see in the above dataframe, if a site fails, a retest is done, where it can be passed or can still fail. If the failure happens again, again the site is retested.

I want to have the last value of the test from the dataframe for that particular test_num and site_num . So, if certain test_num and site_num for any number of the site appear again in the following rows, the final dataframe should have the last record.

So, the above dataframe should look like this -

==Desired result===

TEST_NUM | HEAD_NUM | SITE_NUM | RESULT
________________________________________ 
10000   |  1          | 0       |  P
________________________________________ 
10000   |  1          | 1       |  P          ----> Replaced the row
________________________________________ 
10000   |  1          | 2      |  P           ----> Replaced the row
________________________________________ 
10000   |  1          | 3       |  P 
________________________________________ 

Ideally, the rows should be in their correct order. Like for any test_num, site 0, then 1, then number 2, then 3.

If the last records from a particular site cannot come in the original order(, or if it would be too messy), the following result would also do.

==Result which can also do the trick===

TEST_NUM | HEAD_NUM | SITE_NUM | RESULT
________________________________________ 
10000   |  1          |  0       |  P
________________________________________ 
10000   |  1          |  3       |  P 
________________________________________ 
10000   |  1          |  1       |  P         ----> Not in the correct order but ok
________________________________________ 
10000   |  1          |  2      |  P          ---->  Kept the last, not in the original order but ok.

What I have tried --

I have tried to maintain 3 variables(old_site, old_site, old_testnum) while parsing above dataframe from the text file. During the creation of each row from the text file, I check if the current site_num is same as the old_site value and if old_testnum value is same as current testnum value. If this is the case, I popped up the last inserted value from the list(list is used to create dataframe after all the parsing) and then inserted the current value in the list, so only the last value remains. But I made huge assumption that the duplicated value appears right after the original record, which can be seen not the case here[SITE_NUM = 1's repeated value comes after 2 SITE(site 2,3)].

Can anyone suggest any way the desired result can be obtained(Desired result), or the other possible acceptable format? It would be great if any API exist which can make it elegant.

Answer 1

Here is a working example for what you're looking for in the question.

#Reproducing dataframe
df = pd.DataFrame() 
df['TEST_NUM'] = [10000,10000, 10000, 10000, 10000, 10000] 
df['HEAD_NUM'] = [1,1,1,1,1,1]
df['SITE_NUM'] = [0,1,2,3,1,2]
df['RESULT'] = ['P', 'F', 'F', 'P', 'P', 'P'] 

df = df.drop_duplicates(['TEST_NUM', 'SITE_NUM'], keep='last')
df = df.sort_values('SITE_NUM')

I have just read your comment, and from what I understand you have another column 'test_txt' is an extra column and you wish to remove duplicates and then sort, primarily using the 'test_txt' column.

df = df.drop_duplicates(['test_txt', 'TEST_NUM', 'SITE_NUM'], keep='last')
df = df.sort_values('SITE_NUM')
df = df.sort_values(['test_txt', 'SITE_NUM'])

If this is not what you are looking for please update your question with further detail.