Dynamic Index in ElasticSearch from Logstash

Question

I have following configuration in logstash whereby I am able to create dynamic "document_type" into ES based on input JSON received:

 elasticsearch {
                hosts => ["localhost:9200"]
                index => "queuelogs"
                document_type => "%{action}"
            }

Here, " action " is the parameter that I receive in JSON and different document_type gets created as per different action received.

Now I want this to be done same for Index creation, such as following:

elasticsearch {
            hosts => ["localhost:9200"]
            index => "%{logtype}"
            document_type => "%{action}"
        }

Here, " logtype " is the parameter that I receive in JSON.

But somehow in ES, it creates index as "%{logtype}" only, not as per actual logtype value .

The input JSON is as following:

{
  "action": "UPLOAD",
  "user": "123",
  "timestamp": "2016 Jun 14 12:00:12",
  "data": {
    "file_id": "2345",
    "file_name": "xyz.pdf"
  },
  "header": {
    "proj_id": "P123",
    "logtype": "httplogs"
  },
  "comments": "Check comments"
}

Here, I tried to generate index in following ways:

• index => "%{logtype}"

• index => "%{header.logtype}"

But in both the cases, Logstash does not replace the actual value of logtype from JSON.

Answer 1

You need to specify it like this:

elasticsearch {
        hosts => ["localhost:9200"]
        index => "%{[header][logtype]}"
        document_type => "%{action}"
    }