为什么用空函数运行std :: thread会花费大量内存

Question

I wrote a simple program which should run two threads, sort small array (~4096 Byte) and write into an output file. Input data contain in the one big file (~4Gb). Computer has 128MB memory. I found that running just empty main function use 14MB memory. If run std::thread with empty function application start to use ~8MB per thread. BUT if i make just one dynamic memory allocation program starts to use approximately 64Mb per thread. I don't understand what can spend so much memory. How can I control this size? And how allocate dynamic memory to minimize some system default allocation?

• System: Ubuntu 14.04.3
• Compiler: gcc 4.8.4

• Compiler option:' -std=c++11 -O3 -pthread '

• This is a code example

   void dummy(void) { std::vector<unsigned int> g(1); int i = 0; while( i<500000000) { ++i; } } int main(void) { std::thread t1(&dummy); std::thread t2(&dummy); std::thread t3(&dummy); t1.join(); t2.join(); t3.join(); return 0; } 

Answer 1

Every thread has its own stack. On Linux, the default stack size is 8 MB. When you start allocating memory for the first time, the heap memory allocator might actually reserve a big chunk up front. This might explain the 64 MB per thread you are seeing.

That said, when I say "allocated", that doesn't mean that this memory is really used. The allocation happens in the virtual memory space of the process. This is what you see under the column VSZ when you run ps or under the column VIRT when you run top . But Linux knows that you probably are not going to use most of that allocated memory anyway. So while you have allocated a chunk of virtual memory, Linux does not allocate any physical memory to back that up, until the process actually starts writing to that memory. The real physical amount of memory used by a process is seen under RSS for ps and RES for top . Linux allows more virtual memory to be allocated than there is physical memory in total.

Even though you might not run out of physical memory, if you have a lot of threads on a 32-bit system, each of which is allocating 8 MB of virtual memory, you might run out of the virtual memory space of your process (which is in the order of 2 GB). While C++'s thread library does not allow you to change the size of the stack, the C pthreads library allows you to do this by supplying pthread_create() with a pthread_attr_t which you adjusted using pthread_attr_setstacksize() . See also this stackoverflow question .

Answer 2

The value that you reported for ulimit -s in the comments above does suggest that the thread is still allocating a stack even if it is an empty main. The function call that is executed in the thread would require a stack to pass a return address assuming you're on x86.

@Karrek SB is heading in the right direction with this. The allocator that you are using can affect the heap size for your program. In order to avoid repeated calls to brk or sbrk, allocators will usually request larger initial blocks of memory. It isn't unreasonable to expect values in the order of MB -- especially values that align nicely along typical page boundaries such as 4, 8, 32, 64, etc. when the allocator is first initialized.

To control how much memory is allocated, your results may vary. See if your allocator supports the mallopt function. With a bit of trial and error, you may be able to reduce your overall memory footprint. Else, you could always implement your own allocator.