有效处理并发的stl容器扩展的方法

Question

I have this cute little problem that I have an STL container (unordered_map, or vector) that may both be read and expanded by any number of threads, and I want to do it as efficiently as possible (ie reduce the lock latency by as much as possible).

Now, the easy part is to use a shared lock for reading and an exclusive lock for extending, so I might be doing something like this:

boost::shared_lock myLock;
std::unordered_map myMap;

...

myLock.lock_shared();
//try looking up key
myLock.unlock_shared();
if(!success) {
    myLock.lock();
    //retry looking up key
    if(!success) {
        myMap[key] = value;
    }
    myLock.unlock();
}

While I believe this to be rather efficient in the general case, the time the exclusive lock is held may explode if myMap decides that it needs to reallocate its internal storage. The time that lock is held might go up to several hundred microseconds, and it will block any thread that attempts to read the container during this time.

Does anybody know of an idiom that can be used to avoid blocking all reading threads while the storage is reallocated?

Of course, I know the math that tells me that these latencies will not accumulate to bring down overall performance, nevertheless I would be happier if I could limit the possible latencies somehow.

Answer 1

I can see two possible solutions here. But they may be incompatible with other logic of your program.

1) You can preallocate once (or more rarely than STL does it) enough number of container's elements with vector::reserve() / unordered_map.reserve() . If that is suitable for your program then you will avoid further container reallocations or lower their number.

2) You can use the copy-modify-swap approach. It may increase the number of memory allocations and copyings greatly but will also decrease greatly the time when threads are blocked.

The code would be something like this:

atomic<unordered_map*> pmyMap;

void ChangeMap(){
  unordered_map* pOldMap = pmyMap.load();
  unordered_map* pNewMap = 0;
  {
    if (pNewMap != 0) delete pNewMap;
    pNewMap = new std::unordered_map(*pOldMap); // copy
    (*pNewMap)[key] = value; // modify the copy
  }
  while (!pmyMap.compare_exchange_weak(pOldMap, pNewMap)) // swap
}

In that case you can read from the container without the locks at all. The main problem here is to delete old copy of the container correctly. You can't delete it simply at the end of ChangeMap() because other thread can read from it (eg iterate it) at the same time. To delete the old container correctly you have to track the reading threads in some way and delete the container object only when all reads are finished.

Answer 2

Intel's TBB library is very good. I've used concurrent_hash_map.

Answer 3

There are two obvious solutions: reserve enough space before you start, or copy-modify-swap.

Both have significant downsides. If you don't reserve enough, you end up having to occasionally block everything to re- reserve . And copy-modify-swap is overkill usually.

So do both.

Keep track of when you will need a reallocation. When you don't need a reallocation, simply lock and write.

When you will need a reallocation, copy, reallocate, store, then lock&swap in.

Now things get tricky here. If two threads try to store we want the second to block. So we require a two layer system, where writing threads block on the copy-swap if it is happening, and reading threads only block on the lock&swap-back.

boost::shared_lock myLock;
boost::lock lock2;
std::unordered_map myMap;

...

myLock.lock_shared();
//try looking up key
myLock.unlock_shared();
if(!success) {
  lock2.lock(); -- write-exclusive
  if (we have room to add another value) {
    myLock.lock(); -- block-readers
    myMap[key] = value;
    myLock.unlock(); // allow readers back in, time is one write with no realloc
  } else {
    auto myCopy = myMap; // ouch
    myCopy.reserve( 50% more memory, or 2x as much memory, or whatever );
    myCopy[key] = value;
    myLock.lock(); // -- block readers
    using std::swap;
    swap( myCopy, myMap );
    myLock.unlock(); // allow readers back in, time is 1 swap!         
  }
  lock2.unlock();
}

a downside to this is that other writers can have to wait a long time to write if we are doing a realloc.

If you have only one writer thread, lock2 need not exist.