gensim LDA：如何为每个主题生成带有不同词的主题？

Question

I'm using the LDA algorithm from the gensim package to find topics in a given text.

I've been asked that the resulting topics will include different words for each topic, EG If topic A has the word 'monkey' in it then no other topic should include the word 'monkey' in its list.

My thoughts so far: run it multiple times and each time add the previous words to the stop words list.

Since: A) I'm not even sure of algorithmically/logically it's the right thing to do. B) I hope there's a built in way to do it that i'm not aware of. C) This is a large database, and it takes about 20 minutes to run the LDA each time (using the multi-core version).

Question: Is there a better way to do it?

Hope to get some help,

Thanks.

Answer 1

LDA provides for each topic and each word a probability that the topic generates that word. You can try assigning words to topics by just taking the max over all topics of the probability to generate the word. In other words if topic A generates "monkey" with probability 0.01 and topic B generates the word monkey with probability 0.02 then you can assign the word monkey to topic B.

Answer 2

I think what you want to do is logically incorrect. Take for example a word like "bank" which has two different meaning("river bank" or "money bank") depending on the context. When you intentionally remove the word from one topic words it's probable that you lose the topic meaning(specially when the probability of that word is high). Take a look at this:

https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/topic_methods.ipynb

i think the only remaining option(if it's even be rational to do) is to use the probabilities of words in topics.