sem_post does not prioritise sem_wait call in other thread

Question

I am working on my project where it is crucial that everything is cleaned up properly and all buffered log messages are saved to a file etc.. I am calling exit from another thread and I was thinking to use a semaphore to wait for the cleanup in main thread to occur before the program fully exits. The problem is that when I call sem_post from my exit handler registred with atexit, sem_wait will not immediately decrement the semaphore thus sem_wait in my exit handler will exit immediately because the semaphore is greater than zero.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <semaphore.h>
#include <pthread.h>

static sem_t sem;

void *do_work(void *arg) {
  // if something fails
  printf("Something failed!\n");
  exit(1);
}

void exit_handler() {
  sem_post(&sem); // wake up main thread
  sem_wait(&sem); // wait for cleanup in main
  sem_destroy(&sem);
}

int main() {
  pthread_t worker_thread;

  sem_init(&sem, 0, 0);
  atexit(exit_handler);

  pthread_create(&worker_thread, NULL, do_work, NULL);

  sem_wait(&sem); // block this thread until work is done

  // simulate some cleanup
  usleep(1000000);
  printf("This never gets called!\n");

  sem_post(&sem); // destroy semaphore
}

This example demonstrates my problem. Is there any solution to this problem? I mustn't put the cleanup in my exit handler because I have a lot of local resources in the main function that needs to be cleaned up in my actual program.

Answer 1

You can't control which thread will return from sem_wait() after a call to sem_post() . You need to use two semaphores:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <semaphore.h>
#include <pthread.h>

static sem_t wakeupMain;
static sem_t cleanupDone;

void *do_work(void *arg) {
  // if something fails
  printf("Something failed!\n");
  exit(1);
}

void exit_handler() {
  sem_post(&wakeupMain); // wake up main thread
  sem_wait(&cleanupDone); // wait for cleanup in main
  sem_destroy(&wakeupMain);
  sem_destroy(&cleanupDone);
}

int main() {
  pthread_t worker_thread;

  sem_init(&wakeupMain, 0, 0);
  sem_init(&cleanupDone, 0, 0);
  atexit(exit_handler);

  pthread_create(&worker_thread, NULL, do_work, NULL);

  sem_wait(&wakeupMain); // block this thread until work is done

  // simulate some cleanup
  usleep(1000000);
  printf("This never gets called!\n");

  sem_post(&cleanupDone); // destroy semaphore
}