如何同时遍历pandas的列和行？

Question

I have two df A & B, I want to iterate through df B's certain columns and check values of all its rows and see if values exist in one of the columns in A, and use fill null values with A's other columns' values.

df A:

 country region product
 USA     NY     apple
 USA     NY     orange
 UK      LON    banana
 UK      LON    chocolate
 CANADA  TOR    syrup 
 CANADA  TOR    fish

df B:

 country ID    product1     product2     product3     product4     region 
 USA     123   other stuff  other stuff  apple        NA           NA
 USA     456   orange       other stuff  other stuff  NA           NA
 UK      234   banana       other stuff  other stuff  NA           NA
 UK      766   other stuff  other stuff  chocolate    NA           NA
 CANADA  877   other stuff  other stuff  syrup        NA           NA
 CANADA  109   NA           fish         NA           other stuff  NA

so I want to iterate through dfB and for example see if dfA.product ( apple ) is in columns of dfB.product1-product4 if true such as the first row of dfB indicates, then I want to add the region value from dfA.region into dfB's region which now is currently NA.

here is the code I have, I am not sure if it is right:

import pandas as pd 
from tqdm import tqdm


def fill_null_value(dfA, dfB):
    for i, row in tqdm(dfA.iterrows()):
        for index, row in tqdm(dfB.iterrows()):
            if dfB['product1'][index] == dfA['product'][i]:
                dfB['region'] =  dfA['region '][i]

            elif dfB['product2'][index] == dfA['product'[i]:
                dfB['region'] =  dfA['region'][i]

            elif dfB['product3'][index] == dfA['product'][i]:
                dfB['region'] =  dfA['region'][i]

            elif dfB['product4'][index] == dfA['product'][i]:
                dfB['region'] =  dfA['region'][i]

            else:
                dfB['region '] = "not found"


    print('outputing data')
    return dfB.to_excel('test.xlsx')

Answer 1

The main issue here seems to be finding a single column for products in your second data set that you can do your join on. It's not clear how exactly you are deciding what values in the various product columns in df_b are meant to be used as keys to lookup vs. the ones that are ignored.

Assuming, though, that your df_a contains an exhaustive list of product values and each of those values only ever occurs in a row once you could do something like this (simplifying your example):

import pandas as pd

df_a = pd.DataFrame({'Region':['USA', 'Canada'], 'Product': ['apple', 'banana']})
df_b = pd.DataFrame({'product1': ['apple', 'xyz'], 'product2': ['xyz', 'banana']})

product_cols = ['product1', 'product2']

df_b['Product'] = df_b[product_cols].apply(lambda x: x[x.isin(df_a.Product)][0], axis=1)
df_b = df_b.merge(df_a, on='Product')

The big thing here is generating a column that you can join on for your lookup

Answer 2

If i where you I would create some join and then concat them and drop duplicates

df_1 = df_A.merge(df_B, right_on=['country', 'product'], left_on=['country', 'product1'], how='right')
df_2 = df_A.merge(df_B, right_on=['country', 'product'], left_on=['country', 'product2'], how='right')
df_3 = df_A.merge(df_B, right_on=['country', 'product'], left_on=['country', 'product3'], how='right')
df_4 = df_A.merge(df_B, right_on=['country', 'product'], left_on=['country', 'product4'], how='right')

df = pd.concat([df_1, df_2, df_3, df_4]).drop_duplicates()