Combining 2 dataframe using Inner join and taking last observation only in Python

Question

I have 2 dataframes and i want to join them on 2 columns and get only the record if more than 1 record is present for that inner joins

DF1

DF2

When i combine both Dataframe using Inner Join on 'Patient_id' and 'diag_date', I get

I want only idx '934814' of DF1 -> Nasal Steroids to map against '42775' of DF2, and not with any other indexes I dont want to groupby patient_id, and take the last record., it is required while merging the 2 tables. I want only the last row in inner join instead of it applying on all. Can you guys please suggest some solutions!

Thanks a lot!

Answer 1

Use DataFrame.drop_duplicates with keep='last' and columns used for join before DataFrame.merge :

df = (DF1.drop_duplicates(['Patient_id','Prescription_date'], keep='last')
         .merge(DF2, on=['Patient_id','Prescription_date']))