StatsD与Riemann有什么区别？哪一个在大型分布式系统上表现更好？

Question

What is difference between StatsD vs Riemann ? and which one performs better on large scale distributed systems? we have a distributed platform built on Java and we want to monitor application metrics and perhaps some alerts. We understand that instrumentation is not free so Ideally we are looking for a highly scalable application monitoring framework which can add the least instrumentation cost to our platform/apps and be able to do all sorts of aggregations and so on. I also understand we can build something that is a combination of both but I can't think of reasons why? as both seem to do aggregations and so on but I am unable to identify which one would be a better fit or why one performs better than the other. It will be a great help if someone can share their experiences on these tools in the industry.

Answer 1

I don't have hard numbers on statsd, but Github's Brubeck post suggests that they were losing about 40% of their statsd events at--I'm guessing these graphs are in seconds--25,000 events/sec. Their replacement for statsd, in C, is pushing 4.3 million events/sec. http://githubengineering.com/brubeck/

Riemann won't compete with that on a per-packet basis, but in batches of, say, 10-100 metrics/message, I've heard multiple production users report 10 million events/sec. Unlike statsd, Riemann will scale to all available cores--I've saturated both network interfaces and all 48 cores on my box in tests--but actual performance is gonna vary depending on contention and what you do with your streams. Could be much slower. All depends.

Compared to statsd, Riemann has a much richer event model and performs arbitrary computation. A small Riemann config can replicate of statsd's functionality--but Riemann really shines when you need multidimensional rollups, state transition detection, integration with all kinds of other storage and alerting services, flap suppression, flow control, etc etc etc.

The cost of that is working in a programming language--Clojure--that may be unfamiliar to your team, and having to reason more carefully about scope, state, and if you're writing your own streams, concurrency. Riemann also isn't as widely deployed, which could be a drawback in terms of library support and hiring staff.

Answer 2

The best performer would be Brubeck which is a Statsd compatible (written in C) and therefore you can use the same Statsd client libraries to connect to it.

Brubeck is written in C, Statsd is written in Node.js. And as Github explained in their article , they consider Node.js a foreign technology and they gradually replaced any Node.js services they had. One of them was statsd due to performance issues.

The second best in performance would be Riemann (however, it needs its own client libraries). Statsd would be the slowest.