如何测试文本集群应用程序？

Question

I am developing an application to cluster documents according to their topics. I am using the LDA (Latent Dirichlet Allocation) algorithm. Now the prototype is ready and there are some results.

I am looking for a reasonable way to test it. My current approach is to print out the topics and some of their related documents respectively. And manually evaluate them. I can think of the following test points:

• The documents within a topic are on that topic indeed.
• The topics are substantially different from each other.

Is there any best practice to do this? Is there any objective metric for this rather than my subjective evaluation?

Answer 1

1.after training, we get the topic word matrix P(z|w) , every row is the word's prob assign to the topic, so you can print out the top N words of every topic,and eval them , it would be easy comparing to eval topic with document

2.I think the problem you are asking here is whether the training has converged,I simply eval the P(z|w) ,when the P(z|w) is stable , it means model converge at the param (alpha,beta,topic_num)we choose. and when we tune the topic num , we can get the stable P(z|w) respect to all the topic_num, we choose topic_num respect to the max P(z|w) . you can refer to the paper http://psiexp.ss.uci.edu/research/papers/sciencetopics.pdf

3.as to how to tune alpha beta, and effcient way to tune topic_num , Hanna M. Wallach do a lot of research about that,I simply do this by intuition,since the dataset is too large http://people.cs.umass.edu/~wallach/