为什么malloc（）分配的内存比要求的多，并且如何在Mac OS X中禁用malloc？

Question

I am making a program for a robot which stores a Maze as a Multi-Dimensional Dynamic Adjacency Array, and since the nodes will be discovered along the Maze Traversal, I am trying to allocate initial memory for it and then reallocate memory once a new node is found. However, I once realized that I was storing more data than I had allocated using malloc (I had forgotten to use realloc ) and despite that it was not giving me some sort of a segmentation fault or some other error. So I was interested to know:

1. Why does malloc allocate more memory than required?
2. How can I disable this so that I can properly debug my program? I am sure that the robot would not have loads of heap memory like my 16GB RAM MacBook so I would want to know if my program works within the memory I had allocated to it in the program.

Thank you for your response!

Answer 1

Unlike what most people think, malloc does not allocate the memory directly from the kernel. Instead, it takes a chunk of memory from the kernel, and then manages that chunk in user space with a data structure. That data structure used to be a heap, hence calling it "heap memory".

The practical upshot of the above is that the memory allocated using malloc does not have invalid memory around it. Overstepping the memory by a little will not, in fact, cause your program to crash.

In addition to the above, malloc itself does not allocate in any size you wish. It usually allocates chunks in multiples of 8, 16 or 32 bytes, depending on the data structure used to manage the heap. When you allocate 1 byte with malloc, you are actually getting around 15 bytes that will never be used by anyone. Overwriting those bytes will not cause any noticeable ill effect.

If you want to make sure your program works with a small amount of memory, rlimit/ulimit and co are your friends. If you want to make sure you are not overstepping your allocated buffers, I strongly recommend the address sanitizer. Compile (both clang and gcc) with -fsanitize=address , and your program will crash as soon as you overstep your buffers. This comes at a performance cost, of course.

I'll also add that hardware access control are only possible on page boundaries. On Intel, that's 4096 bytes. Even for memory allocated from the kernel, accesses past the end of the allocation will not trigger a segmentation fault if they do not step outside the allocated page.

Answer 2

1. It doesn't, you just got lucky. Your program invoked undefined behavior, so anything could have happened.

2. If you want to run your program in a constrained environment, to simulate the fact that your target machine has limited memory, you can. Before launching your program, in your shell, run this command: ulimit -Sv 1000 . It's in kilobytes, so 1000 means 1 megabyte.

For more on ulimit, see here: http://ss64.com/osx/ulimit.html - you can use it on Linux too.

You can also use the C function setrlimit() to do similar within your program, if you don't feel like running ulimit every time.