Custom heap allocator - the allocated object must point back to allocator?

Question

In environment that has many instances of custom heap allocators,
are the following statements correct? :-

1. Each allocated pointer should be encapsulated in a type (eg Ptr )
   that cache pointer of who allocated me ( Allo* ) eg

    class Ptr={ Allo* alloPtr; //<-- to keep track public: void delete(){ //call back to something like "a.deallocate()" } } Allo a; //a custom allocator Ptr<X> x=a.allocate<X>(); x.delete(); 

   I think I need to do that to make delete() easier.

2. If the allocator is more complex, eg internally keep a lot of chunks :-

   

   Ptr should keep track which chunk it resides on, OR
   there must be some tiny flag before the allocated content :-

   

   Otherwise, I have to iterate each chunk to find the address -> low performance.

Sorry if this is too newbie, I am very new to custom allocators.
It is not homework - I am trying to improve my game library.

I have read :-

• http://www.dice.se/news/scope-stack-allocation/
• http://gamesfromwithin.com/start-pre-allocating-and-stop-worrying
• http://www.swedishcoding.com/2008/08/31/are-we-out-of-memory/
• https://www.gamedev.net/resources/_/technical/general-programming/c-custom-memory-allocation-r3010

Answer 1

There have been implementations where the meta-data is held near the object which is being used, but these systems suffer from security.

If you have control over a buffer overrun, or a buffer underrun, you can patch the memory of the allocators metadata, and result in causing your code to execute instead of the allocation.

I have seen an implementation which uses completely separate memory areas for data and meta-data. This ensures that overruns and underruns, mis-frees can be identified.

The downside of this system, is the searching of the meta-data from the pointer.

This means that some form of container needs to be keyed on the memory, and suffers from the O(ln n) forms of performance which is associated with most container implementations, so is less performant at its management than if the data is local.

Update

In place meta-data

+----------+               +-----------+           +--------------+
| pointer  |               | vtable    | ------->  | int free()   |
| to chunk | ------------->|meta data  |           S void*alloc() S
+----------+               +-----------+
|memory    |
|          |
+----------+

The pointer to the chunk occurs just before, (or after) the returned memory. If memory is used in array of pointers, then memory[-1] is valid memory, but points to the metadata. If from outside the code, I can convince it to write a value there, I have gained control over the program, as it will then look for the vtable which I have control over

Update 2

Let me try again

Each allocated pointer should be encapsulated in a type (eg Ptr) that cache pointer of who allocated me (Allo*) eg

So specifically not necessarily each item has its own metadata.

In order to track allocations, it is important to understand which scheme was used. In modern programming, it is common to have allocations for small items (< 10 bytes) dealt with differently to items which are large (>500 bytes). The small items can be allocated from slots in an array, with only meta-data held for a block.

Larger sizes don't require the same amount of bulking, as the relative cost of the overhead goes down.

I would expect small memory blocks to be batched together, with larger items having individual meta-data.

Ptr should keep track which chunk it resides on, OR there must be some tiny flag before the allocated content :-

Specifically there needs to be a way to find the memory allocation scheme being used for an allocation, and that could be done by storing the meta-data with the allocation, or having some external scheme keyed by the allocation address to find the memory (eg)

std::map<void*, MetaData> mMetadataChunk;

Note the danger of allocating using std::map, which has 2 memory allocations per tracked item.

Simple allocation schemes also exist, adding a (thread-safe) linked-list to a particular class, allows the memory to be re-used easily for a given object

 class FrequentAllocateDestroy {
      ...
      static SomeListType mSpare;
      void* operator new  ( std::size_t count ) {
          // lock 
          if (mSpare.empty ) {
              return ::new( count );
          } else {
              void * mem = mSpare.top();
              mSpare.pop();
              return mem;
          }
      }
      void operator delete( void * mem){
           // lock
           mSpare.push( mem );
      }
 }