集中式日志服务器/创建类似logstash- *的索引

Question

I am trying to configure ELK to take logs form two different applications. To my understanding, the Elasticsearch index is analogous to a to a database. So my question is:

Is it advisable to configure separate indices for different applications and Why? What are the pros and cons from it being used as/for a centralized logging system?

The ES has an inbuilt functionality with the default logstash-* index that it creates an index on a daily basis depending on the timestamp that it receives from the logs. Now how do I create an index for my app so that it is named different, say App1-* that behaves exactly as the logstash-* index ie is gets created on a daily basis from the timestamp?

I checked the ES documentation on index APIs but couldn't find sufficient information for this for a custom index. Any pointers in this?

Answer 1

The major con for me in using multiple indices is that each one (and the underlying shards) chews up HEAP, which limits the number of indexes you can have open at one time. If you combine the data into one index, it will take less memory to keep the data available.

As for the daily indices, elasticsearch will create any index when asked to do so. Logstash, in the elasticsearch{} output, allows you to specify the name of the index, which can contain static ("logstash-") and dynamic elements (date, fields from the event, etc).

Be aware that there is a mapping template that is applied to the "logstash-*" indices. If you need any of that functionality, you'll need to handle it yourself.