Multiple Logstash instances vs Filebeats

Question

I'm trying to establish the best architecture for our elastic stack implementation.

We have two distinct networks (lets call them internal and external) and several web / db / application servers (approx 10) on each of these networks.

I would like to consume IIS logs, our rabbitMQ messages and some other bits and bobs from machines in both networks and send them to a single server on the internal network where my elastic and kibana installation are located.

For the servers on both the internal and external networks I can see two main ways to get the logs sent to elastic.

1. Setup logstash on each server and send the output to the elastic server on the internal network.
2. Setup filebeats on each server and send the logs to a single server running logstash (this could be the same box that hosts elastic and kibana)

I'm unsure of the pros and cons of these approaches at the moment. I believe the correct approach is to use Filebeats, but I'm unaware why I wouldn't just put logstash in multiple places as it seems like I would be better distributing the processing of logs. Then again, perhaps having one logstash with 20-30 inputs isn't a problem?

Interested in any thoughts or guidance in this area.

Answer 1

From what I read in the documentation, Logstash is much more demanding in term of memory than Filebeat, especially if you do some kind of treatment on the logs (like grok parsing). Logstash represent at least a JVM (with JRuby). For filebeat, I assume its footprint is much smaller, since it's optimized for shipping logs (I never used it, so I can't say).

Also it complicates any update you would want to do to the Logstash instances or their configurations.

For a centralized Logstash, the advantage would be that it is easy to change the adress of the Elasticsearch instance, redirect to a cache like redis or add another output. I also found Logstash (in version 2.+) required frequent restart, so that's easier if you only have one instance to deal with.

I have never used Logstash with multiple inputs, so I can't say.

---

In the job where I was responsible of a log centralisation system, we used beaver (a filebeat equivalent) to ship the logs to a redis server and we had two or three Logstash server sending everything to Elasticsearch. All of the comments above comes from that period.