word2vec：训练语料库中的顺序排列

Question

I have a question concerning the word2vec algorithm. In fact, my question is if the order of the sentences in the training corpus is important. For example, given two training corpus:

CorpusA: Sentence 1. Sentence 2. Sentence 3.

CorpusB: Sentence 3. Sentence 1. Sentence 2.

Will the results from word2vec be different?

Thanks in advance

Answer 1

Order of sentences would impact the embedding learnt from the text corpus since most word2vec implementations are trained using SGD.

So answer to your question - yes, results of word2vec be different.

I don't think word2vec is the right algorithm to use if order of sentences in the corpus is important to you. Keep in mind, output of word can vary because of multiple reasons, few of which are -

• random initialisation of vectors
• negative sampling
• multi-threading
• floating-point precision of your machine

For better results, we do multiple epochs over the training data which won't be possible in your case