如何使用 FastText 查找相似词？

Question

I am playing around with FastText , https://pypi.python.org/pypi/fasttext ,which is quite similar to Word2Vec . Since it seems to be a pretty new library with not to many built in functions yet, I was wondering how to extract morphological similar words.

For eg: model.similar_word("dog") -> dogs. But there is no function built-in.

If I type model["dog"]

I only get the vector, that might be used to compare cosine similarity. model.cosine_similarity(model["dog"], model["dogs"]]) .

Do I have to make some sort of loop and do cosine_similarity on all possible pairs in a text? That would take time...!!!

Answer 1

使用 Gensim，使用 load.word2vec 模型加载 fastText 训练好的 .vec 文件，并使用 most_similiar() 方法找到相似的单词！

Answer 2

You can install pyfasttext library to extract the most similar or nearest words to a particualr word.

from pyfasttext import FastText
model = FastText('model.bin')
model.nearest_neighbors('dog', k=2000)

Or you can get the latest development version of fasttext, you can install from thegithub repository :

import fasttext
model = fasttext.load_model('model.bin')
model.get_nearest_neighbors('dog', k=100)

Answer 3

You should use gensim to load the model.vec and then get similar words:

m = gensim.models.Word2Vec.load_word2vec_format('model.vec')
m.most_similar(...)

Answer 4

You can install and import gensim library and then use gensim library to extract most similar words from the model that you downloaded from FastText .

Use this:

import gensim
model = gensim.models.KeyedVectors.load_word2vec_format('model.vec')
similar = model.most_similar(positive=['man'],topn=10)

And by topn parameter you get the top 10 most similar words.

Answer 5

Use gensim,

from gensim.models import FastText

model = FastText.load(PATH_TO_MODEL)
model.wv.most_similar(positive=['dog'])

More info here

Answer 6

Fasttext has a method called get_nearest_neighbors. nearest neighbor queries . One needs the model's.bin file to use this.