将一个 dataframe 中的一列与另一个 dataframe pandas 中的许多列进行比较

Question

I have two dataframes:

df1:

     ID       name1
0    ''    'company-1'
1    ''    'company2'
2    ''    'company 3'

df2:

     ID      name2       name3        name4
0    '1'   'company1'  'company.1'  'company-1'
1    '2'   'company2'  'company.2'  'company-2'

I want to compare df1['name1'] to the name columns in df2 and put the ID in df2 in the ID column in df1.

I did this:

for i in range(len(df1)):
    for j in range(len(df2)):
        if df1.iloc[i]['name1'] == df2.iloc[j]['name2']:
            df1.iloc[i]['ID'] = df2.iloc[j]['ID']
            break
        elif df1.iloc[i]['name1'] == df2.iloc[j]['name3']:
            df1.iloc[i]['ID'] = df2.iloc[j]['ID']
            break
        elif df1.iloc[i]['name1'] == df2.iloc[j]['name4']:
            df1.iloc[i]['ID'] = df2.iloc[j]['ID']
            break
        else:
            df1[i]['ID'] = ''

Expected result would be:

     ID       name1
0    '1'    'company-1'
1    '2'    'company2'
2    ''    'company 3'

It works, but it's extremely inneficient and takes up to hours. Can you please help me?

I'm sorry if the question doesn't meet the required criteria. It's my first time posting here. I would love some tips regarding that also.

Answer 1

This can be tackled in many ways. You can use a row-wise apply , convert the second frame into a mapping/lookup table (Python dict ), or try joining the two frames. Here's an example of the latter:

import pandas as pd

# The given input data
data_1 = {"ID": ["", "", ""], "name1": ["company-1", "company2", "company 3"]}
data_2 = {"ID"   : ["1", "2"], "name2": ["company1", "company2"], "name3": ["company.1", "company.2"],
          "name4": ["company-1", "company-2"]}

df_1 = pd.DataFrame(data_1)
df_2 = pd.DataFrame(data_2)

# Changing the second frame into "long format" and only keeping the "ID" and "potential_matches" variables
unpivoted: pd.DataFrame = df_2.melt("ID", value_name="potential_matches")[["ID", "potential_matches"]]

# Merging and tidyiing up
expected = (df_1
            .merge(unpivoted, how="left", left_on=["name1"], right_on=["potential_matches"])
            .drop(columns=["ID_x", "potential_matches"])
            .rename(columns={"ID_y": "ID"})[["ID", "name1"]])

print(expected)

If performance is still a problem, you can try matching name1 on a multi-index of name2, name3, name4 .

Output

ID	name1
1	company-1
2	company2
nan	company 3