Is there any way to see virtual memory usgae of a process per thread?

Question

I am using RHEL 6.3(64-bit). I made some changes in my multithreaded c/c++ program (used std:map for a specific requirement) and now initial virtual memory usage of my project has gone really high(around 900m). All the changes i made are guarded under a MACRO.

But after disabling the MACRO also i can not see any reduction in the virtual memory usage of my process.

So i have couple of question to ask.

1. Is there any way to find out which thread is consuming maximum virtual memory so that i can isolate the issue?

2. Why after disabling the macro virtual memory usage of my process is not going down?

3. The same project on 32-bit (RHEL-6.3) is taking really less virtual memory(around-150m). Almost 6 time lesser than 64 bit OS. why?

   I used top command to see virtual memory usage.

Answer 1

You probably should use valgrind to hunt memory leaks . Of course compile all your program with debug info and warnings (eg g++ -Wall -g ). With a recent GCC compiler (eg gcc-4.9 ) on x86-64 you could also use the address sanitizer ( -fsanitize=address )

When running (for tests) your program, you could use pmap(1) to query its address space. See also proc(5) . Try also htop . Benchmark the original program (without any of yours modifications) on the 64 bits system.

^{I hope that you are using a version control system like git on your code and your modifications.}

By definition, the address space is common to all threads in your process . It is wrong to think that some particular thread is consuming memory - it is the process itself (ie any thread within the running program).

Perhaps you want to compile for the x32 ABI .

BTw, each thread needs some memory (for stack, TLS, etc...) -which of course is visible to other threads, eg if you pass them some pointer inside it- and probably more on 64 bits than on 32 bits. How many threads do you have? (You probably should avoid having more than a dozen threads).

Answer 2

64-bit code pointers are 8-bytes instead of the 4-bytes with 32-bit code. So if your code uses lots of pointers, it is easy to double your heap size. But that does not explain 6x bigger.

Sounds like you want to find out where your memory usage is going. Google for "memory profiling" to find a tool.

I usually do memory profiling myself in my code

extern void * operator new( size_t, const char *, int );

#if 1
#define MY_NEW() new
#else
#define MY_NEW() new( __FILE__, __LINE__ )
#endif

Then change all use of 'new' to MY_NEW() in your code and change the #if when you want to enable memory profiling. The operator new() calls malloc() and prints all 3 arguments to a file. Then you make a script to slice and dice the data how ever you want. Adding up all the allocations for one FILE & __LINE_ combination is usually a good place to start. This method does not cover memory allocations in libraries (like STL).

Answer 3

It is useful to profile your program's memory consumption with valgrind --tool=massif ( http://valgrind.org/docs/manual/ms-manual.html ) and check its memory consumption before and after your modifications.

2) First, since you say "I used top command to see virtual memory usage" it is is quite useful to measure all memory in a process ( http://valgrind.org/docs/manual/ms-manual.html#ms-manual.not-measured ) valgrind --tool=massif --pages-as-heap=yes your-program and check in a generated report where the peak of memory consumption was and stacktraces of biggest allocations.

2) Second, if on the step 1 you understand that the problem is in heap allocations check your application with valgrind --tool=massif your_program and compare memory consumption of processes before and after modification.

Answer 4

you can try ps command grep for your process and list the threads ps -eLF , i couldn't recollect exactly to sort by memory usage.

try top -- command 'H' Option to show thread