简体   繁体   中英

Calculate matching percentage of 2 dataframes in python

When joining df2 on df1 , using First_Name , Last_Name and Email , how can I calculate the percentage of df2 that can be matched to df1 ?

df1:

    First_Name  Last_Name   Email                     Value1
0   Aaron       Potter      aaronpotter@gmail.com     10
1   Bella       Granger     bellagranger@gmail.com    2
2   Ron         Black       black@hotmail.com         20
3   Harry       Weasley     harryweasley@hotmail.com  11

df2 :

    First_Name  Last_Name   Email                     Value2
0   Aaron       Potter      aaronpotter@gmail.com     10
1   Ronald      Black       ronaldblack@hotmail.com   5
2   Bella       Granger     bellagranger@gmail.com    2
3   Harry       Weasley     tomriddle@hotmail.com     20

For example, percentage of match is 2 out of 4 in this case.

@anky has a great solution to the problem. I will offer the indicator parameter in merge to visually see the matches.

df_out = df1.merge(df2, on = ['First_Name', 'Last_Name', 'Email'], 
          indicator='Matched', how='out')
df_out

Output:

  First_Name Last_Name                     Email  Value1  Value2     Matched
0      Aaron    Potter     aaronpotter@gmail.com    10.0    10.0        both
1      Bella   Granger    bellagranger@gmail.com     2.0     2.0        both
2        Ron     Black         black@hotmail.com    20.0     NaN   left_only
3      Harry   Weasley  harryweasley@hotmail.com    11.0     NaN   left_only
4     Ronald     Black   ronaldblack@hotmail.com     NaN     5.0  right_only
5      Harry   Weasley     tomriddle@hotmail.com     NaN    20.0  right_only

Or, left-join:

df_out = df1.merge(df2, on = ['First_Name', 'Last_Name', 'Email'], 
          indicator='Matched', how='left')
print(df_out)

Output:

  First_Name Last_Name                     Email  Value1  Value2    Matched
0      Aaron    Potter     aaronpotter@gmail.com      10    10.0       both
1      Bella   Granger    bellagranger@gmail.com       2     2.0       both
2        Ron     Black         black@hotmail.com      20     NaN  left_only
3      Harry   Weasley  harryweasley@hotmail.com      11     NaN  left_only

And using, @anky's solution:

(df_out['Matched'] == 'both').sum()/df_out.shape[0]

Output:

0.5

@Scott Boston's answer is perfect, If you only have 'First_Name', 'Last_Name' and 'Email'. you can use the following code.

df = pd.concat([df1[['First_Name','Last_Name','Email']],df2[['First_Name','Last_Name','Email']]])
df = df.reset_index(drop=True)
gb = df.groupby(list(df.columns))
idx = [x[0] for x in gb.groups.values() if len(x) == 2]
df.reindex(idx)

    First_Name  Last_Name   Email
0   Aaron   Potter  aaronpotter@gmail.com
1   Bella   Granger bellagranger@gmail.com

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM