多个索引logstash模板

Question

Since we are using different servers with different purposes, we configured FIlebeat to send the logs to one specific Index per server. Beta Index:Beta Test Index:Test Build Index:Jenkins

All the filters are working properly but trying to get the .raw fields I believed the templates were not configured properly. Now I'm facing a hard time because it seems that everyone only use logstash- index

My output is:

output {
  elasticsearch {
    hosts => ["172.31.28.8:9200"]
    manage_template => false
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
    document_type => "%{[@metadata][type]}"
  }
}

Additionally, I installed the filebeat template, but again, it is waiting for "filebeat-" index and not for the one I have created.

Answer 1

I have the feeling that it is because of your manage_template = false setting. Are you creating an index template of your own? If not, try allowing Elasticsearch to manage the template for you.

https://www.elastic.co/guide/en/logstash/current/plugins-outputs-elasticsearch.html#plugins-outputs-elasticsearch-manage_template

The .raw fields are not_anazlyed versions of the field. The default setting for a string mapping is analyzed (while the default setting for long, date, etc is not_analyzed). Strings by default have an analyzed main field and a not_analyzed (.raw) field.

https://www.elastic.co/guide/en/elasticsearch/reference/current/default-mapping.html