為什么我在Gensim word2vec中獲得單字母詞匯？

Question

我正在構建一個word2vec模型，如下所示。

from gensim.models import word2vec, Phrases
documents = ["the mayor of new york was there", "human computer interaction and machine learning has now become a trending research area","human computer interaction is interesting","human computer interaction is a pretty interesting subject", "human computer interaction is a great and new subject", "machine learning can be useful sometimes","new york mayor was present", "I love machine learning because it is a new subject area", "human computer interaction helps people to get user friendly applications"]
sentence_stream = [doc.split(" ") for doc in documents]

bigram = Phrases(sentence_stream, min_count=1, delimiter=b' ')
trigram = Phrases(bigram[sentence_stream], min_count=1, delimiter=b' ')

for sent in sentence_stream:
    bigrams_ = bigram[sent]
    trigrams_ = trigram[bigram[sent]]

    print(bigrams_)
    print(trigrams_)


# Set values for various parameters
num_features = 10    # Word vector dimensionality                      
min_word_count = 1   # Minimum word count                        
num_workers = 4       # Number of threads to run in parallel
context = 5          # Context window size                                                                                    
downsampling = 1e-3   # Downsample setting for frequent words


model = word2vec.Word2Vec(trigrams_, workers=num_workers, \
            size=num_features, min_count = min_word_count, \
            window = context, sample = downsampling)

vocab = list(model.wv.vocab.keys())
print(vocab[:10])

但是，我得到的模型詞匯輸出是單個字符，如下所示。

['h', 'u', 'm', 'a', 'n', ' ', 'c', 'o', 'p', 't']

我正確地得到了雙胞胎和三卦。 因此，我只是把代碼弄錯了。 請告訴我這是什么問題？

Answer 1

這解決了我的問題。 我應該將列表列表傳遞給word2vec模型，如下所示。

trigram_sentences_project = []


bigram = Phrases(sentence_stream, min_count=1, delimiter=b' ')
trigram = Phrases(bigram[sentence_stream], min_count=1, delimiter=b' ')


for sent in sentence_stream:
    #bigrams_ = [b for b in bigram[sent] if b.count(' ') == 1]
    #trigrams_ = [t for t in trigram[bigram[sent]] if t.count(' ') == 2]
    bigrams_ = bigram[sent]
    trigrams_ = trigram[bigram[sent]]
    trigram_sentences_project.append(trigrams_)