ElasticSearch, how to order aggregations by other field?

Question

I have Q&A project, and I want to implement Elasticsearch in a section called Feed.

This section is a sort of last activity feed.

This is the feed table:

id | question_id | user_id | action_type  | date_added
---------------------------------------------------------------
26 | 29          | 32      | new_answer   | 2017-04-22 18:34:56
36 | 38          | 35      | new_answer   | 2017-04-24 19:42:40
5  | 52          | 25      | new_question | 2017-04-03 16:28:43
2  | 52          | 20      | new_answer   | 2017-05-05 13:22:41

So, with Elasticsearch I wan't to get the data grouped by question_id and order by id DESC.

So I did this:

{
  "size": 0,
  "query": {
    "match_all": {}
  },
  "aggs": {
    "questions": {
      "terms": {
        "field": "question.id",
        "order": {
          "_term": "desc"
        }
      }
    }
  }
}

And I get this result:

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 41,
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "questions" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 10,
      "buckets" : [ {
        "key" : "64",
        "doc_count" : 4
      }, {
        "key" : "63",
        "doc_count" : 5
      }, {
        "key" : "62",
        "doc_count" : 4
      }, {
        "key" : "61",
        "doc_count" : 5
      }, {
        "key" : "60",
        "doc_count" : 1
      }, {
        "key" : "59",
        "doc_count" : 1
      }, {
        "key" : "58",
        "doc_count" : 3
      }, {
        "key" : "57",
        "doc_count" : 3
      }, {
        "key" : "56",
        "doc_count" : 3
      }, {
        "key" : "55",
        "doc_count" : 2
      } ]
    }
  }
}

What can I do to get the questions ordered by id or date_added ?

Thanks

Answer 1

You can get your documents grouped into buckets by question_id and sorted within each bucket by id or date_added using a top hits sub-aggregation.

Here's an example that builds on your aggregation and sorts the documents within each bucket by id in descending order:

{
  "size": 0,
  "aggs": {
    "questions": {
      "terms": {
        "field": "question_id",
        "order": {
          "_term": "desc"
        }
      },
      "aggs": {
        "question_docs": {
          "top_hits": {
            "size": 10,
            "sort": [
              {
                "id": {
                  "order": "desc"
                }
              }
            ]
          }
        }
      }
    }
  }
}

Assuming your mapping for date_added specifies the date field datatype, then you can also substitute date_added for id in the top_hits aggregation. If you let Elasticsearch determine the mapping for you, it's possible that your dates are being stored as text (for Elasticsearch 5.x) or string (anything before 5.x). I indexed the sample data in your question using Elasticsearch 5.4 with dynamic mapping; it set the mapping for your dates as both text (for full-text search, accessed using date_added ) and keyword (for sorting and aggregations, accessed using date_added.keyword ).

You can use the get mapping API to see check the mappings for your index. For example, to see the mappings for index <index_name> , use the following:

curl -XGET "http://localhost:9200/<index_name>/_mapping"