如何使用InvokeHTTP从Apache Nifi执行ElasticSearch agg查询？

Question

I want to execute the following query from Apache Nifi:

GET /myindex/_search
{
  "size": 0,
  "aggs": {
    "range": {
        "date_range": {
            "field": "Datetime",
            "ranges": [
                { 
                   "from": "2017-02-17T16:00:00Z||-1H/H",
                   "to": "2017-02-17T16:00:00Z||/H" 
                }
            ]
        }
    }
  }
}

I want to get the value of doc_count .

I tried to use InvokeHTTP and directly pasted the above-defined query into the field HTTP Method . I also set Remote URL to http://localhost:9200 . I connected InvokeHTTP with PutFile in order to save a response on the disk. The relationship is Response .

When I run InvokeHTTP , it does not give me any error. However, it neither outputs any result (FlowFile for Response). I am sure that the result is not an empty set, because I tested this query with curl .

What is wrong with my approach? Should I define HTTP Method in some different way?

Answer 1

For verbs like POST, PUT, and PATCH, you'd normally have the above JSON body as the content of a flow file and pass that to InvokeHttp, setting the correct verb and URL there. However the InvokeHttp documentation states that the message body will not be sent for a GET verb.

The good news is that the Elasticsearch REST API for the search endpoint supports both GET and POST. From their latest doc: "Both HTTP GET and HTTP POST can be used to execute search with body. Since not all clients support GET with body, POST is allowed as well."

I would set the content of a flow file to the above JSON body (perhaps using GenerateFlowFile or ReplaceText), then use POST as the verb.

Answer 2

There is a JsonQueryElasticsearch processor, which supports JSON elasticsearch queries.

Your query goes into the Query field. The processor has hits and aggregations relations for you to process the result.