std::conditional_variable::notify_all does not wake up all the threads

Question

I have a simple example here:

The project can be called academic since I try to learn c++11 threads. Here is a description of what's going on.

Imagine a really big std::string with lot's of assembly source code inside like

mov ebx,ecx;\\r\\nmov eax,ecx;\\r\\n....

Parse() function takes this string and finds all the line positions by marking the begin and the end of the line and saving those as string::const_iterators in a job queue.

After that 2 worker threads pop this info from the queue and do the parsing of a substring into an Intstuction class object. They push_back the resulted instance of Instruction class into the std::vector<Instruction> result

Here is a struct declaration to hold the line number and the iterators for a substring to parse

struct JobItem {
    int lineNumber;
    string::const_iterator itStart;
    string::const_iterator itEnd;
};

That's a small logger...

void ThreadLog(const char* log) {
    writeMutex.lock();
    cout << "Thr:" << this_thread::get_id() << " " << log << endl;
    writeMutex.unlock();
}

That's the shared data:

queue<JobItem> que;
vector<Instruction> result;

Here are all the primitives for sync

condition_variable condVar;
mutex condMutex;
bool signaled = false;

mutex writeMutex;
bool done=false;
mutex resultMutex;
mutex queMutex;

Per-thread function

void Func() {
    unique_lock<mutex> condLock(condMutex);
    ThreadLog("Waiting...");
    while (!signaled) {
        condVar.wait(condLock);
    }
    ThreadLog("Started");
    while (!done) {
        JobItem item;
        queMutex.lock();
        if (!que.empty()) {
            item = que.front(); que.pop();
            queMutex.unlock();
        }
        else {
            queMutex.unlock();
            break;
        }
        //if i comment the line below both threads wake up
        auto instr = ParseInstruction(item.itStart, item.itEnd);
        resultMutex.lock();
        result.push_back(Instruction());
        resultMutex.unlock();
    }

The manager function that manages the threads...

vector<Instruction> Parser::Parse(const string& instructionStream){
    thread thread1(Func);
    thread thread2(Func);

    auto it0 = instructionStream.cbegin();
    auto it1 = it0;
    int currentIndex = instructionStream.find("\r\n");
    int oldIndex = 0;
    this_thread::sleep_for(chrono::milliseconds(1000)); //experimental 


    int x = 0;
    while (currentIndex != string::npos){
        auto it0  = instructionStream.cbegin() + oldIndex;
        auto it1  = instructionStream.cbegin() + currentIndex;

        queMutex.lock();
        que.push({ x,it0,it1 });
        queMutex.unlock();
        if (x == 20) {//fill the buffer a little bit before signal
            signaled = true;
            condVar.notify_all();
        }
        oldIndex = currentIndex + 2;
        currentIndex = instructionStream.find("\r\n", oldIndex);
        ++x;
    }
    thread1.join();
    thread2.join();
    done = true;

    return result;
}

The problem arises in the Func() function. As you can see, I'm using some logging inside of it. And the logs say:

Output:
Thr:9928 Waiting...
Thr:8532 Waiting...
Thr:8532 Started

Meaning that after the main thread had sent notify_all() to the waiting threads, only one of them actually woke up. If I comment out the call to ParseInstruction() inside of Func() then both threads would wake up, otherwise only one is doing so. It would be great to get some advice.

Answer 1

Suppose Func reads signaled and sees it false.

Then Parse sets signaled true and does the notify_all ; at this point Func is not waiting, so does not see the notify.

Func then waits on the condition variable and blocks.

You can avoid this by putting a lock of condMutex around the assignment to signaled .

This is the normal pattern for using condition variables correctly - you need to both test and modify the condition you want to wait on within the same mutex.