如何比较两个列表并返回列表中单词的最高相似度

Question

I have a list

list1 = ['good']

I have another list with synonyms of the word "good"

list2 = ['unspoilt', 'respectable', 'honorable', 'undecomposed', 'goodness', 'well', 'near', 'commodity', 'safe', 'dear', 'just', 'secure', 'in_force', 'practiced', 'trade_good', 'proficient', 'expert', 'good', 'sound', 'soundly', 'effective', 'in_effect', 'beneficial', 'dependable', 'unspoiled', 'estimable', 'salutary', 'adept', 'full', 'ripe','upright', 'skilful', 'right', 'serious', 'skillful', 'thoroughly','honest']

Now i wanted to list the word with maximum similarity Is it possible?

suppose if the word good has a similarity greater than 0.8 then i wanted to return those words alone in a list

here let me consider unspoilt has similarity around 0.9

max_similar_list = ['unspoilt']

Answer 1

Here I am using the concept of probability .Those word that have the higher probability of word in a list is the highest similar word in the list .

Try this code ! I am also attach the screenshot of the output .

from difflib import SequenceMatcher

def similar(a, b):
    return SequenceMatcher(None, a, b).ratio()

list1 = ['good']
list2 = ['unspoilt', 'respectable', 'honorable', 'undecomposed', 'goodness', 'well', 'near', 'commodity', 'safe', 'dear', 'just', 'secure', 'in_force', 'practiced', 'trade_good', 'proficient', 'expert', 'good', 'sound', 'soundly', 'effective', 'in_effect', 'beneficial', 'dependable', 'unspoiled', 'estimable', 'salutary', 'adept', 'full', 'ripe','upright', 'skilful', 'right', 'serious', 'skillful', 'thoroughly','honest']
max_similar_list =[]
max=similar(list1[0],list2[1])
max_similar_list=list2[0]
for i in range(1,len(list2)):
  if (max < similar(list1[0],list2[i]) ):
    max=similar(list1[0],list2[i])
    max_similar_list=list2[i]
print("The highest similarity of words in list is '" , max_similar_list , "' with the probabilty of " , max)

Answer 2

For this, you need to define some way to find similarity between a set of words. One way to do this can be Word2Vec which generates word embeddings. Gensim has a good implementation of word2vec, read more here :

https://radimrehurek.com/gensim/models/word2vec.html

For word2Vec, you need corpora to train the model and then make vector embeddings for the given set of words. Then you find the word closest to it using any distance function (eg cosine)

Here is a sample code :

#imports
from nltk.corpus import brown
import numpy as np
from gensim.models import Word2Vec

#Using brown corpus (category news) from nltk. Replace by your corpus with suitable words/sentences
sentences =brown.sents(categories = 'news')

#initialize and train model
model = Word2Vec(min_count=1)
model.build_vocab(sentences)
model.train(sentences, total_examples=model.corpus_count, epochs=model.iter)

# find similarity between two words
model.wv.similarity('good','well')

0.99978923463065106

PS : Here, I'm comparing two words, you can use other methods too which give you most similar word from the corpus. Be careful about words not in corpus.