Elasticsearch can't read ScriptedMetric from BucketScript?

Question

I should work since the ScriptedMetric is a metric, and it does return a single numeric value, but I can't get it to work.

I'm using NEST (5.5.0 via NuGet, I'm targeting Elasticsearch 6.0.0) in C# to get it to work, I did however also try building the same query in Kibana to rule out an issue with NEST. And in Kibana I'm getting exactly the same error.

The error:

buckets_path must reference either a number value or a single value numeric metric aggregation, got: org.elasticsearch.search.aggregations.metrics.scripted.InternalScriptedMetric

My code:

ISearchResponse<LogItem> aggregationResponse = await client.SearchAsync<LogItem>(s => s
    .Size(0)
    .Type("errordoc")
    .Query(...)
    .Aggregations(a => a
    .Terms("Hash", st => st
        .Field(o => o.messageHash.Suffix("keyword")).OrderDescending("Avg-Score")
            .Aggregations(aa => aa
                .Terms("Friendly", ff => ff
                    .Field(oo => oo.friendly.Suffix("keyword"))
                )
                .Max("Max-DateTime", ff => ff
                    .Field(oo => oo.dateTimeStamp)
                )
                .Average("Avg-Score", sc => sc
                    .Script("_score")
                )
                .ScriptedMetric("Urgency-Level", sm => sm 
                    .InitScript(i => i.Inline("params._agg.data = []").Lang("painless"))
                    .MapScript(i => i.Inline("params._agg.data.add(doc.urgency.value)").Lang("painless"))
                    .CombineScript(i => i.Inline("int urgency = 0; for (u in params._agg.data) { urgency += u } return urgency").Lang("painless"))
                    .ReduceScript(i =>i.Inline("int urgency = 0; for (a in params._aggs) { urgency += a } return urgency").Lang("painless"))
                )
                .BucketScript("finalScore", scb => scb
                    .BucketsPath(bp => bp
                        .Add("maxDateTime", "Max-DateTime")
                        .Add("avgScore", "Avg-Score")
                        .Add("urgencyLevel", "Urgency-Level")
                    )
                    .Script(i => i.Inline("params.avgScore").Lang("painless"))
                )
            )
       )
    )
);

The ScriptedMetric aggregation is returning an 11 with my data set.

Am I doing something wrong? Or Is this not possible? If not possible, what would be an alternative?

Also, I know this ScriptedMetric does pretty much do what a Sum would do, but that's going to change of course...

Index mapping:

PUT /live
{
  "mappings": {
    "errordoc": {
      "properties": {
        "urgency": {
          "type": "integer"
        },
        "dateTimeStamp": {
          "type": "date",
          "format": "MM/dd/yyyy hh:mm:ss a"
        }      }
    }
  }
}

Test data:

POST /live/errordoc
{ "messageID": "M111111", "messageHash": "1463454562\/-1210136287\/-1885530817\/-275007043\/-57589585", "friendly": "0", "urgency": "1", "organisation": "Organisation Name", "Environment": "ENV02", "Task": "TASK01", "Action": "A12", "dateTimeStamp": "11\/29\/2017 10:24:21 AM", "machineName": "DESKTOP-SMOM9R9", "parameters": "{ " }

Copy this document a couple of times, maybe changing the urgency/dateTimeStamp, as long as the Hash stays the same it should reproduce my environment...

Answer 1

Somebody on the Elasticsearch forums replied after I created 2 GitHub issues... I did try everything, including this solution, but I must have done something else wrong during that try... Well whatever...

The solution

Change:

.Add("urgencyLevel", "Urgency-Level")

to:

.Add("urgencyLevel", "Urgency-Level.value")