Keep the most recent values and drop older rows (pandas)

Question

I have a dataframe table below which contains new and old values. I would like to drop all the old values while keeping the new values.

ID    Name     Time    Comment
0     Foo   12:17:37   Rand
1     Foo   12:17:37   Rand1
2     Foo   08:20:00   Rand2
3     Foo   08:20:00   Rand3
4     Bar   09:01:00   Rand4
5     Bar   09:01:00   Rand5
6     Bar   08:50:50   Rand6
7     Bar   08:50:00   Rand7

As such it should look like this:

ID    Name     Time    Comment
0     Foo   12:17:37   Rand
1     Foo   12:17:37   Rand1
4     Bar   09:01:00   Rand4
5     Bar   09:01:00   Rand5

I tried to use the code below but this removes 1 new and 1 old value.

df[~df[['Time', 'Comment']].duplicated(keep='first')]

Can anyone provide a correct solution?

Answer 1

I think you can use this solution with to_timedelta , if need filter by max value of column Time :

df.Time = pd.to_timedelta(df.Time)
df = df[df.Time == df.Time.max()]
print (df)
   ID Name     Time Comment
0   0  Foo 12:17:37    Rand
1   1  Foo 12:17:37   Rand1

EDITed solution is similar, only added groupby :

df = df.groupby('Name', sort=False)
       .apply(lambda x: x[x.Time == x.Time.max()])
       .reset_index(drop=True)
print (df)
   ID Name     Time Comment
0   0  Foo 12:17:37    Rand
1   1  Foo 12:17:37   Rand1
2   4  Bar 09:01:00   Rand4
3   5  Bar 09:01:00   Rand5

Answer 2

You can merge group's maximums back to original DF:

df['Time'] = pd.to_timedelta(df['Time'])

In [35]: pd.merge(df, df.groupby('Name', as_index=False)['Time'].max(), on=['Name','Time'])
Out[35]:
   ID Name     Time Comment
0   0  Foo 12:17:37    Rand
1   1  Foo 12:17:37   Rand1
2   4  Bar 09:01:00   Rand4
3   5  Bar 09:01:00   Rand5

Explanation:

In [36]: df.groupby('Name', as_index=False)['Time'].max()
Out[36]:
  Name     Time
0  Bar 09:01:00
1  Foo 12:17:37