Memory pool: force allocating resources (map / vector) to reallocate for continuity

Question

I have a memory pool of fixed size. A custom allocator is used to put std::vector and std::unordered_map elements into that pool. Over time the memory pool gets fragmented and even though there might be enough memory available if you scrap all the free pieces together, it might not be accessible as one single contiguous block as required for std::vector.

Question: How can I defragment this pool (by shifting buckets and vector arrays around) and update all pointers held by std::vector and std::unordered_map to reflect new contiguous memory locations? Is it possible somehow? Other question: Is it sensible?

Answer 1

In general, fragmentation is a hard problem without a clear solution, except switching to a compacting garbage collector.

If your use case involves allocating many same-sized objects, consider separating their allocation into a dedicated bin and allocate memory for that bin with few large allocations. At least this will keep the small objects clustered together rather than spreading out over the whole memory arena. You can have a look at how the Linux kernel does it with its Slab allocator .

There is also a very interesting paper that uses memory mapping tricks to merge fragmented memory. Check out Powers, Bobby, et al. "Mesh: Compacting memory management for C/C++ applications." Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 2019.

Other workarounds exist. std::list and std::deque with their mostly small, same-sized allocations are able to operate under conditions of extreme memory fragmentation longer than std::vector since their allocations are less likely to fail.

Also, you may be able to implement something like memory compacting on an application level: Separate your memory pool in two. Periodically (or when required) invoke an operation that recursively copies all existing objects into new containers, switching memory pools in the process. A poor man's garbage collector.