how does libxml2 deal with fragmentation

Question

I have added my custom allocators for libxml2 in a project and I see a list of alloc and free calls. What I am trying to check is how does libxml2 deal with fragmentation, if does at all. The program executable starts increasing memory and doesnt return it back to OS. There is a libxml2 documentation stating that this might happen in case the OS decides the memory being returned is too small and too much overhead for reusing it. My program uses a lot of small allocations using and freeing them. Rest of the application uses Thread based memory pools so there is no chance of leaking over there, these pools get reset on every iteration of handler calls. I have used the following functions to overwrite the default allocators just to check the allocations. Any help understanding this might be helpful. I have tried searching the website for memory fragmentation dealing with no success.

/* the function override the libxml2 default ones **/
void cfreeFunc(void* ptr);
void * cmallocFunc(size_t size);
void * cmallocAtomicFunc(size_t size);
void * creallocFunc(void *ptr, size_t size);
char * cstrdupFunc(const char *str);

xmlGcMemSetup(cfreeFunc, cmallocFunc, cmallocAtomicFunc, creallocFunc,
        cstrdupFunc);

Answer 1

libxml2 provides you with method to override it's memory management functions - what, if not this, could be done to address your issue? It's up to your functions side - and if you're using sbrk (or malloc which uses sbrk) - you're never returning memory. You could try using separate mmap for every single allocation, however in case of big amount of very small allocs it would significantly increase memory footprint. Better approach is to allocate buffers with mmap and use them as memory pads for future allocations.