Profile cpu and memory usage during py.test execution

Question

I am facing the following project. I want to create pictures showing the cpu and memory level during the execution of a set of performance test suites. I would appreciate any suggestion.

Currently the only approach I am considering is to use the command top or the python module psutil and execute them in parallel to the tests. However I was wondering whether there exists already a better approach, may be a py.test plugin.

A nice to have would be be able to compare those parameters from one execution to another.

The tests are executed Under Linux (Ubuntu).

Answer 1

There are a whole bunch of ways of doing this, ranging from getting broad system statistics and then averaging them (top), to using processor hardware counters (eg using Intel VTune).

psutil seems perfectly fine. My only comment is to make sure you take many measurements and then average them to get rid of spurious spikes and such.

Some other possible ways of taking these measurements are /proc/[pid]/stat (see man page), time , or if you get really obsessive, you can use some programatic techniques, eg for Windows .

Here's a good discussion about programmatically getting benchmarking values. It also discusses some of the traps you can get into, which you should be familiar with even if you are not using a programmatic method.

Intel has a lot of good information about processor benchmarking; it's their bread and butter.

The only other comment I can make is that you need to select your benchmark carefully. Intel emphasizes CPU because it is what they are best at. The same is true for other companies. In truth, there are a whole host of other important factors that come into play depending upon the application domain.

Look at the different media-based benchmarks. They may be more appropriate than one simply based upon processor time. I can't readily find the benchmarks but bing is a wonder.

Answer 2

There are two useful tools for line-by-line timing and memory consumption for functions:

• line profiler
• memory profiler

Installations are easy

$ pip install line_profiler memory_profiler

To do the profiling, decorate your function with @profile , and then run

$ python -m memory_profiler example.py or $ python -m line_profiler example.py