Gensim中的Word2Vec使用model.most_similar

Question

我是Gensim的“ Word2Vec”的新手。 我想为文本建立一个Word2Vec模型（摘自Wikipedia：机器学习），并找到与 “机器学习” 最相似的词 。

我当前的代码如下。

# import modules & set up logging
from gensim.models import Word2Vec

sentences = "Machine learning is the subfield of computer science that, according to Arthur Samuel, gives computers the ability to learn without being explicitly programmed.[1][2][verify] Samuel, an American pioneer in the field of computer gaming and artificial intelligence, coined the term machine learning in 1959 while at IBM. Evolved from the study of pattern recognition and computational learning theory in artificial intelligence,[3] machine learning explores the study and construction of algorithms that can learn from and make predictions on data[4] – such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions,[5]:2 through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance is difficult or infeasible; example applications include email filtering, detection of network intruders or malicious insiders working towards a data breach,[6] optical character recognition (OCR),[7] learning to rank, and computer vision."
# train word2vec on the sentences
model = Word2Vec(sentences, min_count=1)
vocab = list(model.wv.vocab.keys())
print(vocab[:10])

但是，对于vocab，我得到一个字符输出。

['M', 'a', 'c', 'h', 'i', 'n', 'e', ' ', 'l', 'r']

请通过使用model.most_similar帮助我获取most_similar_words

Answer 1

Word2Vec类期望其sentences语料库是各个项目的可迭代来源，每个项目都是一个单词标记列表。

您要提供一个字符串。 如果对其进行迭代，则会得到单个字符。 然后，如果尝试将这些单个字符解释为令牌列表，它仍然只会得到一个字符-因此，它看到的唯一“单词”是单个字符。

至少，您希望您的语料库更像这样构造：

sentences = [
    "Machine learning is the subfield of computer science that, according to Arthur Samuel, gives computers the ability to learn without being explicitly programmed.[1][2][verify] Samuel, an American pioneer in the field of computer gaming and artificial intelligence, coined the term machine learning in 1959 while at IBM. Evolved from the study of pattern recognition and computational learning theory in artificial intelligence,[3] machine learning explores the study and construction of algorithms that can learn from and make predictions on data[4] – such algorithms overcome following strictly static program instructions by making data-driven predictions or decisions,[5]:2 through building a model from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms with good performance is difficult or infeasible; example applications include email filtering, detection of network intruders or malicious insiders working towards a data breach,[6] optical character recognition (OCR),[7] learning to rank, and computer vision.".split(),
]

那仍然只是一个“句子”，但是它将在空白处拆分为单词令牌。

还要注意，有用的word2vec结果需要大量不同的文本样本-玩具大小的示例通常不会显示word2vec以创建而闻名的单词相似性或单词相对排列。