在无锁实现中没有互斥的condition_variable

Question

I have a lock-free single producer multiple consumer queue implemented using std::atomics in a way similar to Herb Sutters CPPCon2014 talk.

Sometimes, the producer is too slow to feed all consumers, therefore consumers can starve. I want to prevent starved consumers to bang on the queue, therefore I added a sleep for 10ms . This value is arbitrary and not optimal. I would like to use a signal that the consumer can send to the producer once there is a free slot in the queue again. In a lock based implementation, I would naturally use std::condition_variable for this task. However now in my lock-free implementation I am not sure, if it is the right design choice to introduce a mutex , only to be able to use std::condition_variable .

I just want to ask you, if a mutex is the right way to go in this case?

Edit: I have a single producer, which is never sleeping. And there are multiple consumer, who go to sleep if they starve. Thus the whole system is always making progress, therefore I think it is lock-free. My current solution is to do this in the consumers GetData Function:
std::unique_lock<std::mutex> lk(_idleMutex); _readSetAvailableCV.wait(lk);

And this in the producer Thread once new data is ready:
_readSetAvailableCV.notify_all();

Answer 1

If most of your threads are just waiting for the producer to enqueue a resource, I'm not that sure a lock-free implementation is even worth the effort. most of the time, your threads will sleep, they won't fight each other for the queue lock.

That is why I think (from the amount of data you have supplied), changing everything to work with a mutex + conditional_variable is just fine. When the producer enqueues a resource it notifies just one thread (with notify_one() ) and releases the queue lock. The consumer that locks the queue dequeues a resource and returns to sleep if the queue is empty again. There shouldn't be any real "friction" between the threads (if your producer is slow) so I'd go with that.

Answer 2

I just watched this CPPCON video about the concurrency TS: Artur Laksberg @cppcon2015

Somewhere in the middle of this talk Artur explains how exactly my problem could be solved with barriers and latches. He also shows an existing workaround using a condition_variable in the way i did. He underlines some weakpoints about the condition_variable used for this purpose, like spurious wake ups and missing notify signals before you enter wait. However in my application, these limitations are no problem, so that I think for now, I will use the solution that I mentioned in the edit of my post - until latches/barrierers are available. Thanks everybody for commenting.

Answer 3

With minimal design change to what you have, you can simply use a semaphore. The semaphore begins empty and is upped every time the produces pushes to the queue. Consumers first try to down the semaphore before popping from the queue.

C++11 does not provide a semaphore implementation, although one can be emulated with a mutex, a condition variable, and a counter. ^†

If you really want lock-free behavior when the producer is faster than the consumers, you could use double checked locking.

/* producer */
bool was_empty = q.empty_lock_free();
q.push_lock_free(x);
if (was_empty) {
    scoped_lock l(q.lock());
    if (!q.empty()) {
        q.cond().signal();
    }
}

/* consumers */
for (;;) {
    if (q.empty_lock_free()) {
        scoped_lock l(q.lock());
        while (q.empty()) {
            q.cond().wait();
        }
        x = q.pop();
        if (!q.empty()) {
            q.cond().signal();
        }
    } else {
        try {
            x = q.pop_lock_free();
        } catch (empty_exception) {
            continue;
        }
        break;
    }
}

Answer 4

One possibility with pthreads is that a starved thread sleeps with pause() and wakes up with SIGCONT . Each thread has its own awake flag. If any thread is asleep when the producer posts new input, wake one up with pthread_kill() .