Potential race condition for a simple multiple threading program

Question

I'm writing a simple program that consists of three threads. Each thread is passed in an object Foo and no matter which thread calls which function, the output for the program will always be "firstsecondthird" . I use semaphore and I'm writing the test code for my implementation. Sometimes, my test case passed but sometimes the test case failed:

input: [1,2,3] = firstsecond
Assertion failed: (false), function test, file /home/foo/printInOrder.cc, line 100.
Abort trap: 6

My program looks like below:

#include "cpputility.h"
#include <functional>
#include <iostream>
#include <semaphore.h>
#include <sstream>
#include <string>
#include <thread>
#include <unordered_map>
#include <vector>

using namespace std;

void printFirst()
{
  cout << "first" << std::flush;
}

void printSecond()
{
  cout << "second" << std::flush;
}

void printThird()
{
  cout << "third" << std::flush;
}

class Foo
{
protected:
  sem_t firstJobDone;
  sem_t secondJobDone;

public:
  Foo()
  {
    sem_init(&firstJobDone, 0, 0);
    sem_init(&secondJobDone, 0, 0);
  }

  void first(function<void()> printFirst)
  {
    printFirst();
    sem_post(&firstJobDone);
  }

  void second(function<void()> printSecond)
  {
    sem_wait(&firstJobDone);
    printSecond();
    sem_post(&secondJobDone);
  }

  void third(function<void()> printThird)
  {
    sem_wait(&secondJobDone);
    printThird();
  }
};

void test()
{
  unordered_map<int, pair<void (Foo::*)(function<void()>), function<void()>>> m({
      {1, {&Foo::first, printFirst}},
      {2, {&Foo::second, printSecond}},
      {3, {&Foo::third, printThird}},
  });
  struct testCase
  {
    vector<int> input;
    string expected;
  };
  vector<testCase> test_cases = {
      {{1, 2, 3}, "firstsecondthird"},
      {{1, 3, 2}, "firstsecondthird"},
  };
  for (auto &&test_case : test_cases)
  {
    std::stringstream buffer;
    std::streambuf *old = std::cout.rdbuf(buffer.rdbuf());
    Foo foo;
    vector<thread> threads;
    for (int i = 0; i < 3; ++i)
    {
      threads.emplace_back(m[i+1].first, foo, m[i+1].second);
    }
    for (auto &&th : threads)
    {
      th.join();
    }
    auto got = buffer.str();
    if (got != test_case.expected)
    {
      printf("input: %s = %s\n",
             CPPUtility::oneDVectorStr<int>(test_case.input).c_str(),
             got.c_str());
      assert(false);
    }
    std::cout.rdbuf(old);
  }
}

int main()
{
  for(int i = 0; i < 10; ++i) {
    // Test repeatedly to detect any potential race condition
    test();    
  }
}

The oneDVectorStr is some helper function I write inside a file called cpputility.h to help print out the 1D vector, here is the implementation to compile the code above

  template <typename T>
  std::string oneDVectorStr(const std::vector<T>& vec) {
    std::string cand = "[";
    for(int i = 0; i < vec.size(); ++i) {
      cand += std::to_string(vec[i]);
      i != vec.size() - 1 ? cand += "," : cand += "";
    }
    cand += "]";
    return cand;
  }

I've stared at this code for quite a while but couldn't locate any race condition. Any suggestion is welcome. Thanks in advance.

Answer 1

I take a further look at code and realize that there is a subtle bug in my test code: I pass Foo object (eg, foo ) by copy when create each new thread. However, I really want to have multiple threads sharing the same Foo object across multiple threads. Thus, I add std::ref to the foo :

threads.emplace_back(m[i + 1].first, ref(foo), m[i + 1].second);

In addition, I print firstJobDone and secondJobDone semaphore values using sem_getvalue() as following:

  Foo()
  {
    sem_init(&firstJobDone, 0, 0);
    sem_init(&secondJobDone, 0, 0);
    int value;
    sem_getvalue(&firstJobDone, &value);
    printf("The initial value of the firstJobDone is %d\n", value);
    sem_getvalue(&secondJobDone, &value);
    printf("The initial value of the secondJobDone is %d\n", value);
  }

And quite shocking, I have:

The initial value of the firstJobDone is 32766
The initial value of the secondJobDone is 32766
input: [1,2,3] = third
Assertion failed: (false), function test, file /home/foo/printInOrder.cc, line 101.
Abort trap: 6

Both semaphores are not properly initialized to 0 with LLVM on the Mac that I'm using. However, my implementation invariant insists that both semaphores have to be initilized to 0 . I don't understand why but I'm assuming since sem_init is marked as deprecated by LLVM, the behavior is not guaranteed to be correct. Thus, per the comments to my question, I change my implementation using conditional variable and mutex, and everything works fine.