How can I check if some rows of a data frame has matches in multiple data frames, sequentially in pandas

Question

I have one dataframe with below data

id	print_volume
A	100
b	200
c	250

Assume the above table represents books in a library. We are going to check if a book is present with any of the 3 readers it has, sequentially. Please note that the column names are all different in these cases.

reader 1

name	volume
c	100
A	120
c	250

reader 2

book	vers
A	100
b	200
c	250

reader 3

book_name	print
p	100
b	200
n	250

Expected output

id	print_volume	present
A	100	2
b	200	3
c	250	1

Here, even though reader 1 and reader 2 has book c with the same volume, we are marking 1 in the present column because we check reader 1, 2 and 3 sequentially. If something is already found, then we don't look further.

This is what I am doing now:

def check_for_book(library_df,reader_df,reader_id):
        subset_to_check=library_df[library_df['present']=='not_found']
        subset_to_check=pd.merge(subset_to_check,reader_df,on=<columns>,how='left',indicator='found')
        subset_to_check['present']=np.where(subset_to_check['found']=='both',reader_id, 'not_found')
        return(pd.concat([subset_to_check,library_df[library_df['present']!='not_found']))

library_df['present']='not_found'
library_df=check_for_book(library_df,reader_df1,'1')
library_df=check_for_book(library_df,reader_df2,'2')
library_df=check_for_book(library_df,reader_df2,'2')

I am not able to find out the bug, the results which I get are inconsistent. Is there a better way to join these data frames in a better way?

Thanks

Answer 1

If you wanna check sequentially row by row, then you can use:

result = []
for n in test.values:
    for i,j,k  in zip(df1.values, df2.values, df3.values):
        if (n == i).all():
            result.append([n[0],n[1],1])
            break
        elif (n == j).all():
            result.append([n[0],n[1],2])
            break
        elif (n == k).all():
            result.append([n[0],n[1],3])
            break
        
final_df = pd.DataFrame(result)

NOTE : Assuming the 1st df name is test and rest 3 - df1 , df2 , df3 .

Answer 2

Let's try something like:

import pandas as pd

df = pd.DataFrame({
    'id': {0: 'A', 1: 'b', 2: 'c'}, 'print_volume': {0: 100, 1: 200, 2: 250}
})

reader1 = pd.DataFrame({
    'name': {0: 'c', 1: 'A', 2: 'c'}, 'volume': {0: 100, 1: 120, 2: 250}
})
reader2 = pd.DataFrame({
    'book': {0: 'A', 1: 'b', 2: 'c'}, 'vers': {0: 100, 1: 220, 2: 250}
})
reader3 = pd.DataFrame({
    'book_name': {0: 'p', 1: 'b', 2: 'n'}, 'print': {0: 100, 1: 200, 2: 250}
})

readers = []
# Rename Columns so they are uniform with df
# Add indicator to each readers
for i, r_df in enumerate((reader1, reader2, reader3)):
    r_df.columns = df.columns
    r_df['present'] = i + 1
    readers.append(r_df)

# Create Readers
readers = pd.concat(readers, axis=0).drop_duplicates(df.columns, keep='first')
# Merge DF and Readers Together
new_df = df.merge(readers, on=df.columns.tolist(), how='left')
print(new_df)

new_df :

  id  print_volume  present
0  A           100        2
1  b           200        3
2  c           250        1

---

Add an Indicator to every Reader so that the DataFrame is identifiable then concat together and drop duplicates so only the first dataframe is kept:

readers :

  id  print_volume  present
0  c           100        1
1  A           120        1
2  c           250        1
0  A           100        2
1  b           220        2
0  p           100        3
1  b           200        3
2  n           250        3