简体繁体 English

用Logstash节流和缩放

[英]Throttling with Logstash and scaling

原文 2017-01-05 08:28:12 8 1 logging/ logstash/ scaling/ elastic-stack/ throttling

I am already using ELK stack for log processing. 我已经在使用ELK堆栈进行日志处理了。 Now, I am planning to use Logstash for throttling of incoming data. 现在，我计划使用Logstash限制传入数据。 But, not sure, How to scale up Logstash + throttling, if load increases. 但是，如果负载增加，则不确定如何扩大Logstash +节流。

Normally for scaling up Logstash, I was planing to run multiple simultaneous instances of Logstash, to share the load. 通常，为了扩大Logstash的规模，我计划同时运行Logstash的多个实例，以分担负载。 But if throttling will be there then how throttling information will be shared among different instances of Logstash ? 但是，如果存在限制，那么如何在Logstash的不同实例之间共享限制信息？ Or any other, completely different approach ? 还是其他完全不同的方法？

1 个解决方案

Based on your additional comments, it sounds like the metrics{} filter would work. 根据您的其他评论，听起来指标{}过滤器可以正常工作。 It allows you to count the number of events that are coming through logstash in a given period, and you could configure different "meter" types based on the different things you wanted to count. 它使您可以计算给定时间段内通过logstash发生的事件数，并且可以基于要计算的不同事物来配置不同的“仪表”类型。

Once you have them counted with that filter, you could have an output{} stanza that would alert you (eg email, hipchat, irc, pagerduty). 使用该过滤器对它们进行计数后，您可能会有一个输出{}节来提醒您（例如，电子邮件，hipchat，irc，pagerduty）。