Application performance metrics for Python

Question

We're writing a Python web-application, where we'd like to know general-ballpark-figures about what our users are doing.

As an example, we have a bunch of legacy-modes for setting the used locale; url-parameter, using different cookies in different formats, Accpet-Language -headers &c. We would really like to measure this in some way (but logging it is simply too much data).

I've tried looking for libraries like Metrics for Node.js , but I can't seem to find anything for Python. Right now, I think we can make do with a bunch of UNIX-load-like counters. (But more is always better, right?)

Have I missed some obvious library or some smart technique somewhere?

Answer 1

刚刚找到了PyCounters项目，看起来就像我需要的那样。

Answer 2

A Python project called Graphite coupled with a Node.js project called Statsd can do exactly the thing you want. Many times people use these projects for software performance metrics and business metrics as well.

Incidentally there are hosted app versions that provide a lot of the same functionality as Graphite and Statsd . My company's product, Instrumental , will do the metric hosting frontend and graph generation for you; in Python, you'd use a Python statsd client in concert w/ our Statsd to Instrumental proxy to report your metrics to our service.

Since the accepted answer was doing file-based logging (by default at least:) ), I'll note the transportation layer is interesting among these different things: A statsd server (whether the official or our proxy ) uses UDP to send your metric information to the server. You can also contact Graphite or Instrumental directly using a line based TCP protocol; other commercial services out there like librato Metrics or Stathat take in your metrics using HTTP messages. Depending on the scale of messages you're sending, any one of these tools may be more or less attractive to you.

Answer 3

You can try a hosting metrics service Metrics At . You can log the data you want to metric, eg url-parameter, Accpet-Language-headers ...etc. Then you can input a regular express in Metrics At. Metrics At will automatically extract data and metric it for you.

Answer 4

One newer alternative to PyCounters is Scales: https://github.com/Cue/scales/

It has a few cool extra features like launching a server to serve current stats or sending to graphite.

Answer 5

Look into multiple logs.

We have many separate logs (separate handles, separate formats) and use the Python logging to log these events into separate logs.

We have lots of functions with things like this.

def some_feature( ... ):
    customer_log.info( "Requested some_feature" )
    ...
    vendor_x_log.info( "Got foo" )
    ...
    vendor_y_log.info( "Got bar" )

Now we have detailed logs. Since the logging module can write to the database (or do any other thing that seems appropriate) we have all the details through a simple, easily-configured interface.