Need to Sort the _term in elastic search aggregation result

Question

Need to Sort the _term in elastic search aggregation result

As per the above question, my elastic search query with the script given by the Val worked fine till the last week. We have upgraded the ES version and suddenly it stopped working.

{
"size": 0,
"aggs": {
"count": {
  "terms": {
    "script": "doc.billingSequence.value as Integer",  <--- transform the terms to integers
    "order": {
      "_term": "asc"
    },
    "value_type": "integer",      <--- consider the terms as integer when sorting
    "size": 10
  }
}
}
}

Now suddenly my ES stopped working with code containing "as Integer" in script. Can anyone please check and help.

[DEBUG] 2015-10-26 10:00:13,907 org.elasticsearch.action.search.type - [Eson the Searcher] [owce_assets][4], node[aNnn7tvHRi6tCkOA-RU2nw], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@123872f5] lastShard [true]
org.elasticsearch.search.SearchParseException: [owce_assets][4]: from[0],size[100]: Parse Failure [Failed to parse source [{"from":0,"size":100,"aggregations_binary":"ewoJCQkJCQkiY291bnQiOiB7IAoJCQkJCQkJInRlcm1zIjogewoJCQkJCQkJCSJzY3JpcHQiOiJkb2MuYmlsbGluZ1NlcXVlbmNlLnZhbHVlIGFzIEludGVnZXIiLAoJCQkJCQkJCSJvcmRlciI6IHsKCQkJCQkJCQkgICJfdGVybSI6ICJhc2MiCgkJCQkJCQkJfSwKCQkJCQkJCQkidmFsdWVfdHlwZSI6ICJpbnRlZ2VyIiwgICAgIAoJCQkJCQkic2l6ZSI6IDIwCgkJCQkJCQkJCgkJCQkJCQkgfQoJCQkJCQl9CgkJCQkJfQ=="}]]
    at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634)
    at org.elasticsearch.search.SearchService.createContext(SearchService.java:507)
    at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:480)
    at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
    at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
    at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
    at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
    at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
    at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)
Caused by: [Error: unknown class or illegal statement: org.elasticsearch.common.mvel2.ParserContext@7915fb96]
[Near : {... doc.billingSequence.value as Integer ....}]
                                         ^

I have tried the given query and now it throws the below exception.

[DEBUG] 2015-10-26 10:33:01,912 org.elasticsearch.action.search.type - [Eson the Searcher] [owce_assets][4], node[aNnn7tvHRi6tCkOA-RU2nw], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest@66accd25] lastShard [true]
org.elasticsearch.search.SearchParseException: [owce_assets][4]: from[0],size[100]: Parse Failure [Failed to parse source [{"from":0,"size":100,"aggregations_binary":"ewoiY291bnQiOiB7CiAgInRlcm1zIjogewogICAgInNjcmlwdCI6ICJkb2MuYmlsbGluZ1NlcXVlbmNlLnZhbHVlIGFzIEludGVnZXIiLAogICAgIm9yZGVyIjogewogICAgICAiX3Rlcm0iOiAiYXNjIgogICAgfSwKICAgICJ2YWx1ZV90eXBlIjogImludGVnZXIiLAogICAgInNpemUiOiAxMCwKICAgICJsYW5nIjogImdyb292eSIgICAgICAgICAgICAKICB9Cn0KfQ=="}]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:507)
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:480)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: script_lang not supported [groovy]
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:143)
at org.elasticsearch.script.ScriptService.search(ScriptService.java:163)
at
org.elasticsearch.search.aggregations.bucket.terms.TermsParser.parse(TermsParser.java:202)
at org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:114)
at org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:77)
at org.elasticsearch.search.aggregations.AggregationParseElement.parse(AggregationParseElement.java:60)
at org.elasticsearch.search.aggregations.AggregationBinaryParseElement.parse(AggregationBinaryParseElement.java:42)
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:622)
... 11 more

The query I am passing as a parameter is::

{
    "count": { 
        "terms": {
            "script": "Integer.parseInt(doc.billingSequence.value)",
                "order": {
                              "_term": "asc"
                            },
                              "value_type": "integer",
                            "size": 20

                         }
                    }
                }

As my indexer contains 1, 2, 4, 6, 14 values in the billingSequence field. But after executing this query, I am getting 2, 4, 6 only in the result.

Exactly my code goes like below..

<mix:variable name="aggregations" as="xs:string">
   <mix:option name="disable-expression-parsing" value="true"/>
       {
            "fatal_errors": {
                "filter": { 
                    "and":[
                        <!-- Filter by date (if given). Value should be send as "`DATE#" and the period should be in Filter data(<Filter name="field_name" value="`DATE#">1d</Filter>).-->
                        <mix:for-each select="//Date">
                            <mix:if test="$comma">,</mix:if>
                                { "range": {
                                    "<mix:value-of select='./@name'/>": {
                                        "gte": "now-<mix:value-of select='./@value'/>",
                                            "lt": "now"
                                        }
                                    }
                                 }
                            <mix:global-variable name="comma" select="true()" as="xs:boolean"/>
                         </mix:for-each>    

                        <!-- Filter all the not nulls or blanks in a field (if given). Value should be send as "`NOT_NULL#" (<Filter name="field_name" value="`NOT_NULL#"></Filter>).-->
                        <mix:for-each select="//NotNull">
                            <mix:if test="$comma">,</mix:if>
                                {
                                    "not":
                                        {
                                            "term":{
                                                "<mix:value-of select='./@name'/>":""
                                            }
                                     }
                                }
                            <mix:global-variable name="comma" select="true()" as="xs:boolean"/>
                        </mix:for-each>                                                 

                        <mix:for-each select="//FitlerTerm">
                            <mix:if test="$comma">,</mix:if>
                                {
                                    "term":{
                                        "<mix:value-of select='./@name'/>":"<mix:value-of select='./@value'/>"
                                    }
                                }
                            <mix:global-variable name="comma" select="true()" as="xs:boolean"/>
                        </mix:for-each> 
                    ]
                },
                "aggs": {
                    "count": { 
                        "terms": {
                            "script": "Integer.parseInt(doc.billingSequence.value)",
                            "order": {
                              "_term": "asc"
                            },
                              "value_type": "integer",
                            "size": 20

                         }
                    }
                }

            }
        }
        <mix:option name="disable-expression-parsing" value="false"/>
</mix:variable>

Here I am passing an aggregation with filter.. My filter is working fine and at the place of aggregation I am getting the issue. Please check and do the needful.

Answer 1

It seems that you're using mvel scripting. You simply need to specify that you want to use groovy instead.

{
"size": 0,
"aggs": {
"count": {
  "terms": {
    "script": "doc.billingSequence.value as Integer",
    "order": {
      "_term": "asc"
    },
    "value_type": "integer",
    "size": 10,
    "lang": "groovy"              <--- add this line
  }
}
}
}

Also make sure that you include this to your project dependencies as it's not included by default:

    <dependency>
        <groupId>org.codehaus.groovy</groupId>
        <artifactId>groovy-all</artifactId>
        <version>2.4.0</version>
        <scope>compile</scope>
        <optional>true</optional>
    </dependency>

UPDATE

If you really cannot use Groovy, you can stick to MVEL and use MVEL type casting instead:

{
"size": 0,
"aggs": {
"count": {
  "terms": {
    "script": "Integer.parseInt(doc.billingSequence.value)",   <--- change this line
    "order": {
      "_term": "asc"
    },
    "value_type": "integer",
    "size": 10
  }
}
}
}