Can HDP (Hierarchical Dirichilet Process) detect the number of topics from the data?

Question

I am using HDP (Hierarchical Dirichilet Process) package from gensim topic modelling software. Gensim HDP implementation expects user to provide number of topics (T) in advance.

hdpmodel.HdpModel(self, corpus, id2word,T=150)

The documentation defines T as top level truncation level.

Can HDP determine number of topics on its own? Is there an implementation of HDP which can detect number of topics? Any help is appreciated.

Answer 1

You can try using the stick-breaking prior .

Some people have reported good results. Others said it did not work for them at all.

Guess you will have to implement this yourself and see if it works.