How to optimize elasticsearch query?

Question

I'm trying to perform an Elasticsearch query by using Java High Level REST Client. The main goal is to group the results for me. Here is a data:

    "hits" : [
  {
    "_index" : "my_index",
    "_type" : "object",
    "_id" : "X4sSPmYB62YwufswHQbx",
    "_score" : 1.0,
    "_source" : {
      "objId" : "1",
      "stepId" : "step_one",
      "status" : "RUNNING",
      "timestamp" : 1515974400
    }
  },
  {
    "_index" : "my_index",
    "_type" : "object",
    "_id" : "15QRP2YB62YwufswAApl",
    "_score" : 1.0,
    "_source" : {
      "objId" : "1",
      "stepId" : "step_one",
      "status" : "DONE",
      "timestamp" : 1516406400
    }
  },
  {
    "_index" : "my_index",
    "_type" : "object",
    "_id" : "QpMOP2YB62YwufswrfYn",
    "_score" : 1.0,
    "_source" : {
      "objId" : "1",
      "stepId" : "step_two",
      "status" : "RUNNING",
      "timestamp" : 1516492800
    }
  },
  {
    "_index" : "my_index",
    "_type" : "object",
    "_id" : "VZMPP2YB62YwufswJ_r0",
    "_score" : 1.0,
    "_source" : {
      "objId" : "1",
      "stepId" : "step_two",
      "status" : "DONE",
      "timestamp" : 1517356800
    }
  },
  {
    "_index" : "my_index",
    "_type" : "object",
    "_id" : "XZMPP2YB62YwufswQfrc",
    "_score" : 1.0,
    "_source" : {
      "objId" : "2",
      "stepId" : "step_one",
      "status" : "DONE",
      "timestamp" : 1517788800
    }
  }
  }
]

For example for objId = 1 I expect to retrieve something like:

    {
      "objId" : "1",
      "stepId" : "step_one",
      "status" : "DONE",
      "timestamp" : 1516406400
    },
    {
      "objId" : "1",
      "stepId" : "step_two",
      "status" : "DONE",
      "timestamp" : 1517356800
    }

Now I have this Java method:

    private List<MyObject> search(String objId) {
    MatchPhraseQueryBuilder queryBuilder = QueryBuilders.matchPhraseQuery("objId", objId);
    SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
    searchSourceBuilder.query(queryBuilder);
    searchSourceBuilder.size(1000);

    SearchRequest searchRequest = new SearchRequest("my_index");
    searchRequest.types("object");
    searchRequest.source(searchSourceBuilder);

    try {
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest);

        return Arrays.stream(searchResponse.getHits().getHits())
                .map(this::toMyObject)
                .collect(toList());
    } catch (IOException ex) {
        log.error("Error retrieving records from elasticsearch. {} ", ex);
    }

    return new ArrayList<>();
}

But this method returns only a list of Objects which are found by objId.

My question is: Is it possible to find objects by objId value than group it by stepId and finally filter that result by the latest timestamp ?

Answer 1

Here is an answer to my question I found:

private List<MyObject> search(String objId) {
    try {
        SearchResponse searchResponse = esRestClient.search(new SearchRequest("my_index")
                .source(new SearchSourceBuilder()
                        .query(QueryBuilders.matchPhraseQuery("objId", objId))
                        .size(0)
                        .aggregation(
                                AggregationBuilders.terms("by_stepId").field("stepId.keyword")
                                        .subAggregation(AggregationBuilders.topHits("by_timestamp")
                                                .sort("timestamp", SortOrder.DESC)
                                                .size(1)
                                        )
                        )
                )
                .types("object")
        );
        Terms terms = searchResponse.getAggregations().get("by_stepId");
        return terms.getBuckets().stream()
                .map(MultiBucketsAggregation.Bucket::getAggregations)
                .flatMap(buckets -> buckets.asList().stream())
                .map(aggregations -> (ParsedTopHits) aggregations)
                .flatMap(topHits -> Arrays.stream(topHits.getHits().getHits()))
                .map(this::toMyObject)
                .collect(toList());
    } catch (IOException ex) {
        log.error("Error retrieving records from elasticsearch. {} ", ex);
    }
    return new ArrayList<>();
}