如何在 pandas 的 merge_asof 中保留重复的“on”列值行

Question

I have two data frames that i want to merge with the merge_asof function but I am facing a small issue.

import pandas as pd

radiology_scan = pd.DataFrame({"patient_id":['a001','a002','a003'],
                     "Visit_year": [2010, 2012, 2016], "scan_report": [1, 55,2]})



OPD_data = pd.DataFrame({"patient_id":['a001','a001','a002','a003','a004','a005'],
                      
                      "Visit_year": [2010 , 2010 , 2013, 2017, 2017,2018],
                      
                      "doctor_comments": ['diagnosis normal, xyz symptoms',
                                          'diagnosis normal, abc symptoms',
                                          'diagnosis abnormal, pqr symptoms',
                                          'diagnosis abnormal, apq symptoms',
                                          'diagnosis normal, xzy symptoms',
                                          'diagnosis abnormal, yzx symptoms' ]})


x = pd.merge_asof(radiology_scan, OPD_data, on =  ["Visit_year"], by =   ["patient_id"] , direction="nearest")

# when using merge_asof only the first visit of the a001's OPD is seen in the merged data frame


y = pd.merge(radiology_scan, OPD_data, on =["Visit_year","patient_id" ], how = 'left' )

# merge function gives both visits of the matched visit year , but i cannot merge the nearest visits of other patients whose
# visit years from both the data frames don't match, hence I want to use merge_asof

When using the merge_asof function I am unable to retain the value of the 2nd visit of the patient a001 from the OPD-data in the merged data frame, even though the id and the visit year matches. The doctor_comment is different and hence i want to retain it in the merged data frame.

On the other hand while using the merge function I am able to retain it but I cannot asof for other patient's data to match the nearest visit year.

The output that I expect is

Answer 1

Reversing the input order may produce what you want, as merge_asof is a "left join" operation:

x = pd.merge_asof(OPD_data, radiology_scan, on=["Visit_year"], by=["patient_id"], direction="nearest")

output:

  patient_id  Visit_year                   doctor_comments  scan_report
0       a001        2010    diagnosis normal, xyz symptoms          1.0
1       a001        2010    diagnosis normal, abc symptoms          1.0
2       a002        2013  diagnosis abnormal, pqr symptoms         55.0
3       a003        2017  diagnosis abnormal, apq symptoms          2.0
4       a004        2017    diagnosis normal, xzy symptoms          NaN
5       a005        2018  diagnosis abnormal, yzx symptoms          NaN

Note that Visit_year is taken from OPD_data in this case.