如何在 Lucene 中突出显示 Boolean FuzzyQueries - 升压必须是正浮点数？

Question

我正在努力为那些打错很多字的用户（比如我自己）提供帮助。

我尝试为一些数据创建一个简单的搜索页面。 我在 BooleanQuery 中构建FuzzyQuery ，因为我希望用户BooleanQuery ，例如：

    BooleanQuery.Builder builder = new BooleanQuery.Builder();

    builder.add(new FuzzyQuery(new Term("body", "pzza")), BooleanClause.Occur.SHOULD);
    builder.add(new FuzzyQuery(new Term("body", "tcyoon")), BooleanClause.Occur.SHOULD);

    BooleanQuery query = builder.build();

搜索按预期工作，但我从 Lucene 8.5 API 文档中获得的用于构建突出显示的代码失败：

    SimpleHTMLFormatter htmlFormatter = new SimpleHTMLFormatter();
    Highlighter highlighter = new Highlighter(htmlFormatter, new QueryScorer(query));
    for (int i = 0; i < hits.length; i++) {
        int id = hits[i].doc;
        Document doc = searcher.doc(id);
        System.out.println("HIT:" +  doc.get("url"));
        String text = doc.get("body");
        TokenStream tokenStream = TokenSources.getAnyTokenStream(searcher.getIndexReader(), id, "body", analyzer);
        TextFragment[] frag = highlighter.getBestTextFragments(tokenStream, text, false, 10);//highlighter.getBestFragments(tokenStream, text, 3, "...");
        for (int j = 0; j < frag.length; j++) {
            if ((frag[j] != null) && (frag[j].getScore() > 0)) {
                System.out.println((frag[j].toString()));
            }
        }
    }

有错误：

java.lang.IllegalArgumentException: boost must be a positive float, got -1.0
    at org.apache.lucene.search.BoostQuery.<init>(BoostQuery.java:44)
    at org.apache.lucene.search.ScoringRewrite$1.addClause(ScoringRewrite.java:69)
    at org.apache.lucene.search.ScoringRewrite$1.addClause(ScoringRewrite.java:54)
    at org.apache.lucene.search.ScoringRewrite.rewrite(ScoringRewrite.java:117)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:246)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.extract(WeightedSpanTermExtractor.java:135)
    at org.apache.lucene.search.highlight.WeightedSpanTermExtractor.getWeightedSpanTerms(WeightedSpanTermExtractor.java:530)
    at org.apache.lucene.search.highlight.QueryScorer.initExtractor(QueryScorer.java:218)
    at org.apache.lucene.search.highlight.QueryScorer.init(QueryScorer.java:186)
    at org.apache.lucene.search.highlight.Highlighter.getBestTextFragments(Highlighter.java:201)

该代码使用了不推荐使用的方法，但我直接从文档中获取。

有人可以解释为什么我会收到此错误吗？ 如何创建与此查询构造一起使用的荧光笔？ 还是我需要不同的Query ？

Answer 1

以下突出显示的方法使用 Lucene v8.5.0 和问题的模糊 boolean 示例。

在我的精简演示中，结果如下所示（当然，您可以优化突出显示的片段的显示方式）：

高亮代码如下：

import java.io.IOException;
import org.apache.lucene.document.Document;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.highlight.SimpleHTMLFormatter;
import org.apache.lucene.search.highlight.Highlighter;
import org.apache.lucene.search.highlight.QueryScorer;
import org.apache.lucene.search.highlight.TokenSources;
import org.apache.lucene.search.highlight.TextFragment;
import org.apache.lucene.search.highlight.InvalidTokenOffsetsException;

public class CustomHighlighter {

    private static final String PRE_TAG = "<span class=\"hilite\">";
    private static final String POST_TAG = "</span>";

    public static String[] highlight(Query query, IndexSearcher searcher,
            Analyzer analyzer, ScoreDoc hit, String fieldName)
            throws IOException, InvalidTokenOffsetsException {
        SimpleHTMLFormatter htmlFormatter = new SimpleHTMLFormatter(PRE_TAG, POST_TAG);
        Highlighter highlighter = new Highlighter(htmlFormatter, new QueryScorer(query));
        int id = hit.doc;
        Document doc = searcher.doc(id);

        String text = doc.get(fieldName);

        TokenStream tokenStream = TokenSources.getTokenStream(fieldName,
                searcher.getIndexReader().getTermVectors(id), text, analyzer, -1);
        int maxNumFragments = 10;

        boolean mergeContiguousFragments = Boolean.TRUE;
        TextFragment[] frags = highlighter.getBestTextFragments(tokenStream,
                text, mergeContiguousFragments, maxNumFragments);

        String[] highlightedText = new String[frags.length];
        for (int i = 0; i < frags.length; i++) {
            highlightedText[i] = frags[i].toString();
        }
        // control how you handle each fragment for display...
        //for (TextFragment frag : frags) {
        //    if ((frag != null) && (frag.getScore() > 0)) {
        //        highlightedText = frag.toString();
        //    }
        //}
        return highlightedText;
    }

}

class 的使用如下（其中SearchResults只是我用于收集结果的类之一，以便稍后呈现给用户）：

for (ScoreDoc hit : hits) {
    String[] highlightedText = CustomHighlighter.highlight(query, searcher,
            analyzer, hit, field);
    String document = searcher.doc(hit.doc).get("path");
    SearchResults.Match match = new SearchResults.Match(document, highlightedText, hit.score);
    results.getMatches().add(match);
}

模糊查询是这样的：

private static Query useFuzzyBooleanQuery() {
    BooleanQuery.Builder builder = new BooleanQuery.Builder();
    builder.add(new FuzzyQuery(new Term("contents", "pzza")), BooleanClause.Occur.SHOULD);
    builder.add(new FuzzyQuery(new Term("contents", "tcyoon")), BooleanClause.Occur.SHOULD);
    return builder.build();
}

上面的代码没有给我任何弃用警告。

我无法解释为什么你会得到那个特定的“提升”错误——我自己没有看到过，也无法重新创建它。 但我并没有太努力，我承认。