Why does the mutex lock twice in LevelDB?

Question

I'm learning the levelDB C++ project, here is the situation, Status s = Write(WriteOptions(), nullptr) triggers a compaction work, and then enter the while loop to wait the signal, the compaction thread goes to BackgroundCall method, and it also needs to lock the mutex_ , I'm not sure the TEST_CompactMemTable still holds the mutex, so I print the debug message in the Lock and Unlock method. The output is just like:

TEST_CompactMemTable mutex lock
BackgroundCallmutex lock
BackgroundCallmutex unlock
TEST_CompactMemTable mutex unlock

I'm confused why does the mutex lock twice? Am i missing something, any help would be greatly appreciated.

Status DBImpl::TEST_CompactMemTable() {
  Status s = Write(WriteOptions(), nullptr);
  if (s.ok()) {
    // lock the mutex first
    MutexLock l(&mutex_);
    while (imm_ != nullptr && bg_error_.ok()) {
      background_work_finished_signal_.Wait();
    }
    if (imm_ != nullptr) {
      s = bg_error_;
    }
  }
  return s;
}

void DBImpl::BackgroundCall() {
  // lock the mutex twice
  MutexLock l(&mutex_);
  assert(background_compaction_scheduled_);
  if (shutting_down_.load(std::memory_order_acquire)) {
  } else if (!bg_error_.ok()) {
  } else {
    BackgroundCompaction();
  }

  background_compaction_scheduled_ = false;

  MaybeScheduleCompaction();
  background_work_finished_signal_.SignalAll();
}

Answer 1

This is simple background_work_finished_signal_ is conditional variable which is associated with a mutex_ .

This is done during construction: see DBImpl::DBImpl

DBImpl::DBImpl(const Options& raw_options, const std::string& dbname)
    : env_(raw_options.env),
      internal_comparator_(raw_options.comparator),
      internal_filter_policy_(raw_options.filter_policy),
      options_(SanitizeOptions(dbname, &internal_comparator_,
                               &internal_filter_policy_, raw_options)),
      owns_info_log_(options_.info_log != raw_options.info_log),
      owns_cache_(options_.block_cache != raw_options.block_cache),
      dbname_(dbname),
      table_cache_(new TableCache(dbname_, options_, TableCacheSize(options_))),
      db_lock_(nullptr),
      shutting_down_(false),
      background_work_finished_signal_(&mutex_),
      mem_(nullptr),
      imm_(nullptr),
      has_imm_(false),
      logfile_(nullptr),
      logfile_number_(0),
      log_(nullptr),
      seed_(0),
      tmp_batch_(new WriteBatch),
      background_compaction_scheduled_(false),
      manual_compaction_(nullptr),
      versions_(new VersionSet(dbname_, &options_, table_cache_,
                               &internal_comparator_)) {}

Now when background_work_finished_signal_.Wait(); is called mutex have to be released, so other thread can lock it and send notifications. When notification is received lock is restored before background_work_finished_signal_.Wait(); returns control.

So basically those logs of yours are from different threads and mutex is unlock and lock by background_work_finished_signal_.Wait(); and your logs do not spot it.

So infact your logs should print something like this:

TEST_CompactMemTable mutex lock
TEST_CompactMemTable_wait_cv mutex unlock
BackgroundCallmutex lock
BackgroundCallmutex unlock
TEST_CompactMemTable_wait_cv mutex lock
TEST_CompactMemTable mutex unlock

Answer 2

When the code execution goes to background_work_finished_signal_.Wait() the mutex gets unlocked and gets locked again after the signal raised. You don't see it, because you don't log messages in the mutex methods. The real flow should be

TEST_CompactMemTable mutex lock
signal wait
TEST_CompactMemTable mutex unlock
BackgroundCallmutex mutex lock
signal raised
BackgroundCallmutex mutex unlock
TEST_CompactMemTable mutex lock
TEST_CompactMemTable mutex unlock