Find Difference between 2 dataframes

Question

I am new to pandas and have a question. I have 2 dataframes with me:

df1 = pd.DataFrame({'ID': ['ID1', 'ID2', 'ID3', 'ID6'],
                 'Value': ['59', '29', '73', '34']})

df2 = pd.DataFrame({'ID': ['ID1', 'ID2', 'ID4'],
                     'Value': ['54', '29', '73']})

I want to get a output dataframe which lists out the changed values (ID1) and individual IDs which are in df1 and df2 (like ID3, ID4 and ID6)

Thanks a lot in advance!

Answer 1

do outer merge and calculate difference:

out=df1.merge(df2,on='ID',how='outer')
out['Difference']=out.pop('Value_x').astype(float)-out.pop('Value_y').astype(float)

output of out :

   ID   Difference
0   ID1     5.0
1   ID2     0.0
2   ID3     NaN
3   ID6     NaN
4   ID4     NaN

OR

After merging use fill NaN's with 0:

out=df1.merge(df2,on='ID',how='outer').fillna(0)
out['Difference']=out.pop('Value_x').astype(float)-out.pop('Value_y').astype(float)

output of out :

    ID  Difference
0   ID1     5.0
1   ID2     0.0
2   ID3     73.0
3   ID6     34.0
4   ID4     -73.0

Answer 2

changed = set()
individual = (set(df1['ID'].to_numpy()) - set(df2['ID'].to_numpy())).union(set(df2['ID'].to_numpy()) - set(df1['ID'].to_numpy()))
for i in set(df1['ID'].to_numpy()) - (set(df1['ID'].to_numpy()) - set(df2['ID'].to_numpy())):
    if not df1[df1['ID'] == i]['Value'].equals(df2[df2['ID'] == i]['Value']):
        changed.add(i)
print(changed, individual)
>>> {'ID1'} {'ID6', 'ID3', 'ID4'}