Measure the maximum memory usage during a function call

Question

I have a Linux/Mac C++ library that performs a series of steps, in the form of function calls. I would like to measure the maximum memory usage during each step.

I am not interested in ad-hoc solutions like starting another thread that polls memory usage, running a profiler, etc.

So far I have found getrusage() is present on Mac and Linux and does return the maximum memory usage, however there appears to be no way to reset this maximum after each function call.

Is there any way around this limitation?

Edit: To be clear, I do not want to commandeer malloc() / free() and log everything. I want a solution that is suitable to keep in running production code.

Answer 1

I had a browse through the Linux source code, and found this :

        /*
         * Writing 5 to /proc/pid/clear_refs resets the peak
         * resident set size to this mm's current rss value.
         */

I haven't tried it yet but it looks promising.

Edit: It was added in this commit

Edit 2: I have looked through the MacOS kernel source - the corresponding value is stored in resident_max . Unfortunately there doesn't seem to be a feature to reset it.

Edit 3: On Linux you can obtain the maximum allocated memory using malloc_info() however there does not appear to be a way to reset it. It also relies on you using glibc .

Answer 2

The malloc and free calls are more than trivial wrappers around sbrk and mmap system calls. This makes getrusage return something that is not in line with calls to malloc and free . Any non-trivial implementation of these functions will manage a free-list inside the process itself, before thinking of returning anything to the system.

The program calls free (or delete for that matter), and the memory is not immediately returned to the operating system (maybe never). The free -ed memory can be reused by the task, if it calls malloc , but not by other processes. This makes getrusage correct from the OS perspective, but not correct from the program perspective.

On Linux you can use mallinfo()

#include <malloc.h>
#include <cinttypes>
std::size_t memsize()
{
    auto data = mallinfo();
    return data.arena - data.fordblks + data.hblkhd;
}

Here, memsize() will return the number of bytes allocated from program's perspective. It takes into account the various allocation techniques, such as sbrk and mmap , and considers rounding up and overhead as part of the allocated memory of malloc() (and new ).

With OSX things are not so bright. You can have a look into the source code of apple's malloc() , and specifically at mstats , which states in the comment:

/*
 * A Glibc-like mstats() interface.
 *
 * Note that this interface really isn't very good, as it doesn't understand
 * that we may have multiple allocators running at once.  We just massage
 * the result from malloc_zone_statistics in any case.
 */

This does not look very promising and:

#include <malloc/malloc.h>
#include <cinttypes>
std::size_t memsize()
{
    auto data = mstats();
    return data.bytes_used;
}

Does not look very good, according to my experiments. But it may be better than nothing, or just relying on getrusage . I think you are out of luck on OSX, unless someone can correct me.