[英]How to add dynamic hosts in Elasticsearch and logstash
I have prototype working for me with Devices sending logs and then logstash parsing it and putting into elasticsearch. 我的原型适用于设备发送日志,然后将logstash解析后放入Elasticsearch。
Logstash output code :- Logstash输出代码 :-
output{
if [type] == "json" {
elasticsearch {
hosts => ["host1:9200","host2:9200","host3:9200"]
index => "index-metrics-%{+xxxx.ww}"
}
}
}
Now My Question is : 现在我的问题是:
I will be producing this solution. 我将提出这个解决方案。 For simplicity assume that I have one Cluster and I have right now 5 nodes inside that cluster.
为了简单起见,假设我有一个集群,并且现在在该集群中有5个节点。
So I know I can give array of 5 nodes IP / Hostname in elasticsearch output plugin and then it will round robin to distribute data. 所以我知道我可以在elasticsearch输出插件中提供5个节点IP /主机名的数组,然后它将轮流分发数据。
How can I avoid putting all my node IP / hostnames into logstash config file. 如何避免将所有节点IP /主机名放入logstash配置文件中。
As system goes into production I don't want to manually go into each logstash instance and update these hosts. 随着系统投入生产,我不想手动进入每个logstash实例并更新这些主机。
What are the best practices one should follow in this case ? 在这种情况下应遵循的最佳实践是什么?
My requirement is : 我的要求是:
I want to run my ES cluster and I want to add / remove / update any number of node at any time. 我想运行我的ES集群,并且想随时添加/删除/更新任意数量的节点。 I need all of my logstash instances send data irrespective of changes at ES side.
我需要所有logstash实例发送数据,而不管ES端的更改如何。
Thanks. 谢谢。
If you want to add/remove/update you will need to run sed or some kind of string replacement before the service startup. 如果要添加/删除/更新,则需要在服务启动之前运行sed或某种形式的字符串替换。 Logstash configs are "compiled" and cannot be changed that way.
Logstash配置是“已编译”的,不能以这种方式更改。
hosts => [$HOSTS] ... $ HOSTS="\\"host1:9200\\",\\"host2:9200\\"" $ sed "s/\\$HOSTS/$HOSTS/g" $config
Your other option is to use environment variables for the dynamic portion, but that won't allow you to use a dynamic amount of hosts. 您的另一个选择是将环境变量用于动态部分,但这将不允许您使用动态数量的主机。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.