shared memory mutex with struct pointers

Question

Im looking for some feedback on if I am doing the following properly. Im working on porting some windows real time code that had heavy use of named mutex's. It took some searching but i came across some stuff saying you can use shared memory as mutexes in linux, using shm open.

I cant include all of the code here, but i put together the key areas that i need some feed back on. My questions are if im setting up the shared memory region, and mutex correctly, and also if my pointers will be set correctly, along with how to utilize it for locking/unlocking.

volatile struct GenQ {
    volatile pthread_mutex_t *qMutexId
    volatile sem_t  qSemId
    volatile int    nexton
    volatile int    nextoff
}

typedef struct Node{
    void                *qid
    char                shmname[80]
    sem_t               *semid
    pthread_mutex_t     *mutexID
    struct node         *next
    struct node         *prev   
}


void * init (const char *qname)
{
    struct GenQ *myq;
    char mtxstr[80];
    pthread_mutex_t *mutexQueAccess;
    int mode = S_IRWXU | S_IRWXG;
    int fd = 0;
    int status = 0;

    mtxstr[0] = "\0";
    strcpy(mtxstr,"/");
    strcat(mtxstr, qname);
    strcat(mtxstr, "_MTX");

    fd = shm_open(mtxstr, O_CREATE | O_RDWR | O_TRUNC, mode);
    if (fd == -1)
        //err handling stuff

    status = ftruncate(fd, sizeof(pthread_mutex_t));
    if(status==0){
        //err handling stuff

    mutexQueAccess = (pthread_mutex_t*) mmap(NULL, sizeof(pthread_mutex_t), 
    PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if(mutexQueAccess == MAP_FAILED)
    //err handling stuff

    pthread_mutexattr_t mutexAttr;
    pthread_mutexattr_init(&mutexAttr);
    pthread_mutexattr_setpshared(&mutexAttr, PTHREAD_PROCESS_SHARED);
    pthread_mutex_init(mutexQueAccess, &mutexAttr);

    myq->qMutexId = mutexQueAccess;

    newNode = (Node*)malloc(sizeof(node));
    newNode->mutedID = mutexQueAccess;
    //add node to link list

}

void * openQ(*const char *qname)
{
    pthread_mutex_t *mutexQueAccess;
    int fd = 0;
    int status = 0;
    char mtxstr[80];
    int mode = S_IRWXU | S_IRWXG;

    mtxstr[0] = "\0";
    strcpy(mtxstr,"/");
    strcat(mtxstr, qname);
    strcat(mtxstr, "_MTX");

    fd = shm_open(mtxstr, O_CREATE | O_RDWR, mode);
        //check fd for err

    mutexQueAccess = (pthread_mutex_t *)mmap(NULL, sizeof(pthread_mutex_t), 
    PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
        //check for err on mmap

    newNode = (Node*)malloc(sizeof(node));
    newNode->mutedID = mutexQueAccess;
    //add node to link list


}

void * enque(const char *qname, char *msg_data)
{
    node = //search for node
    pthread_mutex_lock(&(node->mutexQueAccess))

}

Answer 1

I'm doing the same thing in some code I have, and it looks very similar - in part. The idea is really pretty straight-forward. The size of the space allocated to the mutex is exactly sizeof(pthread_mutex_t) . You create the shared file, mmap it and initialize the mutex in one process, then others can simply mmap the file and set their pthread_mutex_t * pointer to it.

One thing I don't understand is the struct GenQ part. The volatile keywords hint that this structure already resides in shared memory. However, if it is in shared memory, the mutex pointer you're stashing there won't be valid outside the process that created it. The mutex itself may reside in shared memory, and other process(es) may have a pointer to the mutex too, but the exact location of the shared memory region within their virtual address space may differ, hence their pointer value would differ as well (and therefore the pointer should be private to the process).

So if the GenQ structure is already in shared memory, why not simply declare a pthread_mutex_t ( not a pointer -- space for the actual mutex) within the GenQ structure, and then do the same initialization from your master process as you have in the above code. You then don't need a separate shared memory file for the mutex; it will be contained in the same shared memory space as the other queue information.

Also, consider adding the PTHREAD_MUTEX_ROBUST attribute (with pthread_mutexattr_setrobust ) to the mutex as well so that you can recover in the case where one of the processes crashes while holding the lock.

That code looks something like this:

    int err = pthread_mutex_lock(mutex_ptr);
    if (err) {
        if (err == EOWNERDEAD) {
            WARN("Mutex owner died while holding -- recovering");
            err = pthread_mutex_consistent(mutex_ptr);
            if (err)
                 FATAL("Huh?");
        } else {
            FATAL("Wha?");
        }
    }

Lastly, I would quibble with your use of volatile . That is neither necessary nor sufficient. If you access shared variables while you hold the mutex, the volatile qualifier is not needed. If you do not hold the mutex, volatile by itself is insufficient to ensure proper synchronization -- you will still need to use some sort of atomic increments/decrements or atomic compare-and-swap.