An STL implementation that uses a dynamic/state based allocator?

Question

Does anybody know of an STL implementation that allows for dynamic allocators to be passed in to an instance of a container before use.

The scenario is that we have a general memory allocator that manages a number of pools of memory and for each instance of say stl::vector we want to allocate each instance from a different pool of memory.

The problem with the standard STL implementations is that you can only define the memory pool on a type basis ie all vector's of type int would allocate from the same pool.

I've already swapped out our default stl::allocator for one that has a state ie the pool we want to allocate this instance from but this doesn't work well for say stl::list where it allocates things in the default ctor.

For reasons related to our library we also don't have a valid pool in the ctor for all objects and so we want to call a 'set memory pool' function before users can use the stl container.

Has anybody come across an implementation that supports this kind of thing?

Answer 1

I am not so sure about your question precisely. So I'll cover the case of a state-full allocator.

In C++03, any allocator should be able to deallocate resources allocated by another allocator of the same type.

The C++0x standard actually removed this limitation and allows state-full allocators to be passed to STL containers as long as they are Allocator Aware containers (I think it covers all containers packaged with the STL, since they model Sequence ).

For example: [allocator.adaptor] $20.10 Class scoped_allocator is now part of the C++0x STL.

Answer 2

The typed allocator can use a general allocator underneath to perform the allocations.

Allocator needs to support these functions:

  pointer address ( reference x ) const;
  const_pointer address ( const_reference x ) const;
  pointer allocate (size_type n, allocator<void>::const_pointer hint=0);
  void deallocate (pointer p, size_type n);
  size_type max_size() const throw();
  void construct ( pointer p, const_reference val );

Assuming you have a mechanism that just allocates memory and deallocates memory, you can use that to implement some of the functions above.

The advantage of allocators being typed is that you know you are going to create lots of items of exactly the same size and can therefore create your "pages" to fit. The biggest issue can be that you are forced by allocate() to return contiguous buffers (and indeed vector needs them).

http://www.cplusplus.com/reference/std/memory/allocator/

Your question is still a bit unclear as to why this is not sufficient. You can initialise the memory-pool with "once" logic. (If it is multi-threaded you can use boost::once to achieve this).

Answer 3

The problem with the standard STL implementations is that you can only define the memory pool on a type basis ie all vector's of type int would allocate from the same pool.

This is not exactly true. You can have different vectors holding int elements each having a different allocator type.

However, regarding the question --

Does anybody know of an STL implementation that allows for dynamic allocators to be passed in to an instance of a container before use.

-- it is simply not supported by the C++ Standard Library (STL), therefore, while it may be that there are implementations where per-object allocators work, it is not portable.

For a detailed analysis of why and how to use custom allocators see Scott Meyers's book Effective STL , specifically Item 11: Understand the legitimate use of custom allocators .

Answer 4

One option would be to use a thread-local variable to hold the pointer to the memory pool to use, and to capture this in your allocator implementation. For example (using boost::thread_specific_ptr ):

// Global variable
boost::thread_specific_ptr<allocator_impl> real_allocator;

struct set_allocator : boost::noncopyable {
private:
    allocator_impl *old;
public:
    set_allocator(allocator_impl *newAllocator) {
        old = real_allocator.get();
        real_allocator.reset(newAllocator);
    }
    ~set_allocator() {
        real_allocator.reset(old);
    }
};

template<typename T>
struct myallocator {
private:
    real_allocator *delegate;
public:
    myallocator() {
        delegate = real_allocator.get();
    }
    T *allocate(size_t n,  allocator<void>::const_pointer hint=0)
    {
        return delegate->allocate(sizeof(T), n, hint);
    }
    // Other mandatory STL Allocator functions and typedefs follow
};

// later:
allocator_impl *allocator = new allocator_impl();
set_allocator sa(allocator); // Set current allocator using RAII
std::list<int, myallocator> mylist; // using *allocator as the backing allocator

The API that myallocator must implement is described here .

Unfortunately, due to limitations in the STL API, this is as good as you can get without reimplementing the STL. There may, however, be third-party libraries which let you pass in an allocator to the object's constructor out there somewhere.

Answer 5

Ok so STL port does seem to support this kind of functionality, where as Microsoft (VS 2008) and the GNU implementation (a port of stl circa gcc 3.4.1) don't as they allocate/deallocate things in the ctors/dtors.

Here's my test code showing how to do this. Warning its not a full implementation by any means!

#include <list>
#include <assert.h>

namespace my
{

class memory_pool
{
public:
    memory_pool() : m_top( 0 ){};
    ~memory_pool(){};

    void* alloc( int size, int align ) { void* p = (void*)&m_pool[m_top]; m_top += size; return p; }
    void free ( void* p ) { assert( (p >= m_pool) && (p < &m_pool[m_top] ) ); }
private:
    char m_pool[0xFFFF];
    int m_top;
};

template<class T>
class dynamic_allocator 
{
    template<typename U> friend class dynamic_allocator;
public:
    typedef T                   value_type;         
    typedef size_t              size_type;          
    typedef value_type*         pointer;            
    template <class _Tp1> struct rebind { typedef dynamic_allocator<_Tp1> other; };

    dynamic_allocator() : m_pPool( NULL ){}
    dynamic_allocator( memory_pool* pPool ){ m_pPool = pPool; }
    dynamic_allocator( const dynamic_allocator< T >& alloc ) : m_pPool( alloc.m_pPool ){}
    template< typename U >
    dynamic_allocator( const dynamic_allocator< U >& alloc ) : m_pPool( alloc.m_pPool ){}
    ~dynamic_allocator() {}

    pointer allocate( size_type count ){ return allocate( count, NULL ); }
    pointer allocate( size_type count, const void* ) { assert( m_pPool ); return ( pointer )m_pPool->alloc( count * sizeof( T ), __alignof( T ) ); }
    void deallocate( pointer p, size_type count )   { assert( m_pPool ); m_pPool->free( p ); }

    void set( memory_pool* pPool ) { m_pPool = pPool; }

private:
    memory_pool* m_pPool;
};

template< typename T, typename Al = dynamic_allocator<T>  >
class list : public std::list<T, Al>
{
public:
    typedef typename std::list<T, Al>::allocator_type allocator_type;

    list() : std::list<T, Al>(){};
    list( const allocator_type& a ) : std::list<T, Al>( a ){};
    ~list(){};

    void initialise( memory_pool& pool ){ std::list<T, Al>::_M_node.set( &pool ); } // or something like this
    void terminate( void ){ clear(); std::list<T, Al>::_M_node.set( NULL ); }                   // or something like this
};

}; // namespace my

class lemon
{
public:
    lemon(){}       // must be empty ctor as we don't want to have active mem pool in ctor for users to use
    ~lemon(){}

    void initialise( my::memory_pool& pool ){ m_list.initialise( pool ); }
    void terminate( void )                  { m_list.terminate(); }

    void add( float f ) { m_list.push_back( f ); }

private:
    my::list<float> m_list;
};

int main( void )
{
    my::memory_pool poolA;
    my::memory_pool poolB;

    my::dynamic_allocator<float> aa( &poolA );
    my::list<float> a( aa );
    my::list<float> fail;

    std::list<float>::allocator_type bb;
    std::list<float> b( bb );

    a.push_back( 0.2f );
    b.push_back( 50.0f );
    //fail.push_back( 19.0f );

    a.clear();
    b.clear();

    lemon lemons[2];

    lemons[0].initialise( poolA );
    lemons[1].initialise( poolB );

    lemons[0].add( 10.0f );
    lemons[1].add( 20.0f );
    lemons[1].add( 18.0f );

    lemons[0].terminate();
    lemons[1].terminate();

    scanf("press any key\n");

    return 0;
}