计算Lucene文档和质心之间的相似度

Question

In order to perform a simple clustering algorithm on results that I get from Lucene, I have to calculate Cosine similarity between 2 documents in Lucene, I also need to be able to make a centroid document to represent the centroid of each cluster.

All I can think of doing is building my own Vector Space model with tf-idf weighting, using the TermFreqVectors and Overall Term frequencies to populate it.

My question is: This is not an efficient approach, is there a better way to do this?

This feels a little unclear so any suggestions on how I can improve my question are also appreciated.

Answer 1

马克，您可能会发现将Mahout与Lucene集成 ，将IR Math与Java或使用Lucene进行向量空间分类器 集成非常有用。

Answer 2

in order to get similarity of one document to the other, why not make a one query with the content of one document and run query against index? that way, you will get score(cosine similarity values)

Answer 3

The short answer is: No.

I have spent a lot of time (way way too much) looking into this, and as far as I can see, you can make your own Vector Space Model and work from that, or use Mahout to generate a Mahout Vector, which you can make comparisons between documents from. I am gonna go ahead and make my own, so I'm marking this question answered!