Logstash Elasticsearch output use parsed field from input file

Question

I have a lot of old logfiles in the format drupallogYYYYMMDD that I want to throw in logstash and by default the elasticsearch output creates the index for today. I can overwrite that with index: "...." but is there a way to set this in the logstash conf file so it takes the YYYYMMDD from above and turn it into the naming convention logstash-YYYY-MM-DD ?

Answer 1

The index option of the elasticsearch output defaults to "logstash-%{+YYYY.MM.dd}" but you can change it to "logstash-%{+YYYY-MM-dd}" if you prefer hyphens in your date.

The date that gets inserted into said index name pattern is the timestamp of each message (ie the @timestamp) field. Since @timestamp is UTC it might not correspond exactly to the date in your log filename depending on how you name the files and what timezone you're in. You should not attempt to change the timezone of the @timestamp field. Other tools depend on the mapping between a message's @timestamp field and the index in which it's stored.