Elasticsearch highlight fields not in query

Question

I have a document with many fields in it, each field containing short texts.

To enable custom boosting and to keep performance problems at bay I created groupings of these fields according to their boost level. Example:

"partName": {
    "type": "text",
    "copy_to": "fuzzyMatchFields_boost5"
},

This means that to match partName I am going to do a fuzzy matching search on fuzzyMatchFields_boost5 and boost the score 5 times. The query is something like:

QueryBuilders.boolQuery()
    .must(
        QueryBuilders.multiMatchQuery(terms)
            .field("exactMatchFields_boost5", 5f)
            .field("exactMatchFields_boost2", 2f)
            .field("fuzzyMatchFields_boost5", 5f)
            .field("fuzzyMatchFields_boost4", 4f)
            .field("fuzzyMatchFields_boost3", 3f)
            .field("fuzzyMatchFields_boost2", 2f)
    )
...

etc. (I am using the Java client, but that's just btw.)

This works fine, but now I'm trying to add highlighting of matches.

The problem is that I do not query the fields directly, rather than their grouping fields only. So when I try highlighting I get nothing.

I already figured out that I can toggle a flag ( require_field_match ) and just pass the field names regardless:

val highlightBuilder = HighlightBuilder()
    .requireFieldMatch(false)
highlightFields.forEach { field ->
    highlightBuilder.field(field)
}

This works in some cases, but it doesn't work for others. I cannot figure out why certain fields work and why certain fields don't. I'm not even sure if this is supposed to work.

A possible solution is to define another query specifically for the highlighting (this is possible in Elasticsearch). My concerns are around performance, ie. this would obviously require me listing all the fields individually. I don't even understand why I would need a query at all, as I thought that for any matching document the highlighter just processes them individually. But maybe this is not the case.

What is the best solution here?

Answer 1

This did the trick:

Elasticsearch highlighting: "multi-match" query on custom "_all"-fields created with "copy_to"

So I had an n-gram analyzer on some of the aggregated fields, but not on the original ones (because I did not want to query them directly). Once I added the analyzer to the individual fields highlighting started working properly.