how to iterate each row of one dataframe and compare with rows in another dataframe in Python?

Question

I have two dataframes:

DF1:

ID     v1           v2         v3
289  1455.0        2.0        0.62239  
289  1460.0        0.0        0.46037  
289  1465.0        4.0        0.41280 
290  1470.0        0.0        0.39540 
290  1475.0        2.0        0.61809 
290  1475.0        2.0        0.61809

DF2:

ID     v1           v2         v3
289  1423.0        2.0        0.62239  
289  142Q.0        0.0        0.46037  
289  14FW.0        4.0        0.41280  
290  14Q3.0        0.0        0.39540  
290  1453.0        2.0        0.61809 
290  1454.0        2.0        0.61809

I want to iterate each row in DF1 with every row in DF2 and see if it is in DF2, something like:

for row in results_01.iterrows():
    diff = []
    if row not in results_02:
        add different one to 'diff'
        print(diff)

I know the logic but not sure how to do this, new to Python, can anyone help me? Many thanks.

Answer 1

You can do it easily with 'inner' merge.

intersect = pd.merge(df1, df2, how='inner')

Edit:

It turns out that the rows that are in df1 and not in df2 are wanted and not the intersection. In this case one should use the isin pandas method. Here is SO link that deals with it.

Answer 2

One way to do it (maybe not the most efficient) would be to append the dataframes together and then drop duplicates, like so:

full_df = df1.append(df2)
full_df = full_df.drop_duplicates(keep=False)

Answer 3

The code block you have looks pretty close to what you do in python. Take a row from one dataframe and iterate through the other dataframe looking for matches.

for index, row in results_01.iterrows():
    diff = []
    compare_item = row['col_name'] 
    for index, row in results_02.iterrows():
       if compare_item == row['compare_col_name']:
           diff.append(compare_item, row['col_name']
    return diff 

Here I am taking a specific column value from a row from one dataframe and comparing it to another value from the other dataframe