Detached thread won't exit although it runs pthread_exit?

Question

I have been dealing with a problem in a thread pool for some days now. I tried all types of different things but I can't seem to solve the issue. I have made a simple version that reproduces the problem.

Code:

#include <unistd.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h> 


struct bsem_t bsem;
pthread_t threads[2];


/* Binary semaphore */
typedef struct bsem_t {
    pthread_mutex_t mutex;
    pthread_cond_t   cond;
    int v;
} bsem_t;

void bsem_post(bsem_t *bsem) {
    pthread_mutex_lock(&bsem->mutex);
    bsem->v = 1;
    pthread_cond_broadcast(&bsem->cond);
    pthread_mutex_unlock(&bsem->mutex);
}

void bsem_wait(bsem_t *bsem) {
    pthread_mutex_lock(&bsem->mutex);
    while (bsem->v != 1) {
        pthread_cond_wait(&bsem->cond, &bsem->mutex);
    }
    bsem->v = 0;
    pthread_mutex_unlock(&bsem->mutex);
}


/* Being called by each thread on SIGUSR1 */
void thread_exit(){
    printf("%u: pthread_exit()\n", (int)pthread_self());
    pthread_exit(NULL);
}


/* Startpoint for each thread */
void thread_do(){

    struct sigaction act;
    act.sa_handler = thread_exit;
    sigaction(SIGUSR1, &act, NULL);

    while(1){
        bsem_wait(&bsem); // Each thread is blocked here
        puts("Passed semaphore");
    }

}


/* Main */
int main(){

    bsem.v = 0;

    pthread_create(&threads[0], NULL, (void *)thread_do, NULL);
    pthread_create(&threads[1], NULL, (void *)thread_do, NULL);
    pthread_detach(threads[0]);
    pthread_detach(threads[1]);
    puts("Created threads");

    sleep(2);

    pthread_kill(threads[0], SIGUSR1);
    pthread_kill(threads[1], SIGUSR1);

    puts("Killed threads");

    sleep(10);

    return 0;
}

What the code does, is create two threads. Both threads wait on a binary semaphore ( bsem_wait ). Then while they are waiting I send a SIGUSR1 signal to both resulting on pthread_exit() being executed on each thread. On my terminal it shows that everything goes as planned..

Output:

Created threads
Killed threads
2695145216: pthread_exit()
2686752512: pthread_exit()

The problem

Although the output seems correct, using pstree shows that only one of two threads die. The other thread stays alive until the whole program exits. Why is this?

---

Update

Replacing my custom binary semaphore with a normal semaphore seems to solve this for no apparent reason..

#include <unistd.h>
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>
#include <time.h> 
#include <semaphore.h> 

sem_t sem;
pthread_t threads[2];


/* Caller thread will exit */
void thread_exit(){
    printf("%u: pthread_exit()\n", (int)pthread_self());
    pthread_exit(NULL);
}


/* Startpoint for each thread */
void* thread_do(){

    struct sigaction act;
    act.sa_handler = thread_exit;
    sigaction(SIGUSR1, &act, NULL);

    while(1){
        sem_wait(&sem); // Each thread is blocked here
        puts("Passed semaphore");
    }

}


/* Main */
int main(){

    sem_init(&sem, 0, 0); // Normal semaphore

    pthread_create(&threads[0], NULL, thread_do, NULL);
    pthread_create(&threads[1], NULL, thread_do, NULL);
    pthread_detach(threads[0]);
    pthread_detach(threads[1]);
    puts("Created threads in pool");

    sleep(2);

    //PROBLEM
    pthread_kill(threads[0], SIGUSR1);
    pthread_kill(threads[1], SIGUSR1);

    puts("Destroyed pool");

    sleep(10);

    return 0;
}

Answer 1

You can't get there from here

pthread_exit() is not listed in the "signal safe funtions" of the signal(7) man page. rewrite your code to have the pthread_exit call outside of the signal handler.

Answer 2

So the issue seemed to be a deadlock!

The problem is that each thread is waiting inside the bsem_wait of the binary semaphore in different locations:

void bsem_wait(bsem_t *bsem) {
    pthread_mutex_lock(&bsem->mutex); // THREAD 2 BLOCKED HERE
    while (bsem->v != 1) {
        pthread_cond_wait(&bsem->cond, &bsem->mutex); // THREAD 1 WAITING HERE
    }
    bsem->v = 0;
    pthread_mutex_unlock(&bsem->mutex);
}

Thread 1 in this case is the fastest thread. Thread 2 is the slower one. When I run the signal to kill a thread, the waiting thread unblocks and exits as expected. The problem is that it never unlocks the mutex . So the blocked thread (2) remains blocked forever. For some reason, the thread won't be terminated because it is waiting on the mutex.

Just adding an unblock before exiting, solves the issue:

void thread_exit(){
    printf("%u: pthread_exit()\n", (int)pthread_self());
    pthread_mutex_unlock(&mutex); //  NEW CODE
    pthread_exit(NULL);
}

This ofcourse is a hack to demonstrate what is happening and shouldn't be used. I am going to follow Jasen's advice and get rid of the signal handler all together and solve it some other way. Namely I have to assure that the thread goes through the whole bsem_wait !