遍历 dataframe 行以匹配列表中的单词

Question

My goal is to measure similarities between the rows of a dataframe and a list of words. My code looks like this:

import pandas as pd
import distance
import numpy as np
df = pd.DataFrame({'col': ['apps','orange juice','citrs']})
li = ['apple','orange','citrus']
df['SIM'] = np.nan
df['SIM_COL'] = np.nan
for row in df.iterrows():
    row_data = row[1].tolist()
    for l in li:
        if distance.jaccard(row_data[0],l) < 0.5:
            df.loc[[df[df['col']==row_data[0]].index.values[0]],'SIM']= distance.jaccard(row_data[0],l)
            df.loc[[df[df['col']==row_data[0]].index.values[0]],'SIM_COL']= l
            break

And this is my output:

    col SIM SIM_COL
0   apps    NaN NaN
1   orange juice    0.454545    orange
2   citrs   0.166667    citrus

This is fine when i make the distance condition < 0.5 . If i change it to 1 , my output becomes:

    col SIM SIM_COL
0   apps    0.600000    apple
1   orange juice    0.846154    apple
2   citrs   0.900000    orange

Now it gives me the wrong result for orange and citrus. How can i make it so only the lowest distances are considered?

Answer 1

The result is right. See

print(distance.jaccard('orange juice', 'apple'))

# 0.846154

How can i make it so only the lowest distances are considered?

I would use an extra variable min_dist to record the lowest distances. Update df['SIM'] and df['SIM_COL'] , only if the new distance is smaller than the current lowest distances.

for row in df.iterrows():
    row_data = row[1].tolist()
    min_dist = 999  # Init with a big value

    for l in li:
        dist = distance.jaccard(row_data[0], l)
        if dist < 1 and dist < min_dist:
            min_dist = dist
            df.loc[[df[df['col']==row_data[0]].index.values[0]],'SIM'] = dist
            df.loc[[df[df['col']==row_data[0]].index.values[0]],'SIM_COL']= l