简体繁体 English

从Lucene索引中获取最高频率项

[英]Get highest frequency terms from Lucene index

原文 2010-05-12 19:00:40 4 2 java/ lucene/ full-text-search/ indexing/ frequency

i need to extract terms with highest frequencies from several lucene indexes , to use them for some semantic analysis. 我需要从几个lucene索引中提取具有最高频率的术语 ，以便将它们用于某些语义分析。

So, I want to get maybe top 30 most occuring terms(still did not decide on threshold, i will analyze results) and their per-index counts. 所以，我想得到可能排名前30的最常见的术语（仍然没有决定阈值，我将分析结果）和他们的每个索引计数。 I am aware that I might lose some precision because of potentionally dropped duplicates, but for now, lets say i am ok with that. 我知道我可能会失去一些精确度，因为可能会丢失一些副本，但是现在，让我说我很好。

So for the proposed solutions, (needless to say maybe) speed is not important, since I would do static analysis, I would put accent on simplicity of implementation because im not so skilled with Lucene and cant wrap my mind around some concepts of it.. 所以对于提出的解决方案，（不用说可能）速度并不重要，因为我会进行静态分析，我会强调实现的简单性 ，因为我不熟悉Lucene并且无法围绕它的一些概念。。

I can not find any code samples from something similar, so all concrete advices (code, pseudocode, links to code samples...) Appreciate all the advices! 我找不到任何类似的代码示例，所以具体建议（代码，伪代码，代码示例的链接......）感谢所有的建议！

Thank you! 谢谢！

2 个解决方案

A very simple way would be to use Luke . 一个非常简单的方法是使用Luke 。 On the 'Overview' tab, there is a 'Show top terms' button that can be used for what you need. 在“概览”标签上，有一个“显示热门条款”按钮，可用于您需要的内容。

Have a look at this: http://sujitpal.blogspot.com/2009/02/summarization-with-lucene.html 看看这个： http ： //sujitpal.blogspot.com/2009/02/summarization-with-lucene.html

The class in this page has computeTopTermQuery method which you should be easily able to retrofit for going over multiple indexes. 此页面中的类具有computeTopTermQuery方法，您应该可以轻松地对其进行多次索引的改进。