命名共享内存：shm_open没有提供正确的位置

Question

I am having difficulty using shared memory. I believe I am misunderstanding how shared memory is supposed to work. I am trying to create a template class for my project to wrap the POSIX Shared Memory API. When I call mmap() for the second time, in the same process, I am expecting to see the same ptr returned as the first time. This is not what I am seeing. My second address is offset by 0x1000 (I assume this is the page boundary). When I write to the first memory mapped location, the data does not show up in the second.

I thought possibly that the data was not getting syncd, so I tried calling msync(). This did not help.

I am most suspect of the different Addresses returned by mmap(). It seems like this might be a pointer-pointer, but no of the documentation or examples show anything about pointer-pointers. So...

Obviously, this leads me to believe either I am doing something wrong or misunderstand how named shared memory is supposed to work within the same process.

I have been pouring over the man pages and all over SO looking for answers to no avail.

Could some please tell me what I am doing wrong to access the same named shared memory location? Or point me to answer that provides the explanation I require to get me on my way again.

Please Excuse all the debug cout and excessive documentation, I was trying to understand/learn shared_memory API.

NOTE: I understand that there are better ways to share data within the same process. But, this is only a short test driver and this template would be used for in multi-process environment.

EDIT: Im not sure it matters, but I am trying to run on Suse Linux 3.0.101

"Help me Obi-Wan Kenobi, you're my only hope!"

shmem.h

// ****************************************************************************
// POSIX Shared Memory
//     as document by Richard Stevens 
//     "UNIX Network Programming: Interprocess Communications" Vol 2, 2nd Ed.
// -------------------
//
// Shared memory is the fastest form of IPC available, because one copy of the
// data in the shared memory is available to all the threads or processes that
// share the memory. Some form of synchronization is normally required,
// however, to coordinate the threads or processes that are sharing the memory.
//
// POSIX provides two ways to share memory between unrelated processes.
//      1. Memory-mapped files: a file is opened by open, and the resulting
//         descriptor is mapped into the address space of the process by mmap.
//         Memory-mapped files can also be shared between unrelated processes.
//      2. Shared memory objects: the function shm_open opens a POSIX IPC name
//         (perhaps a pathname in the filesystem), returning a descriptor that
//         is then mapped into the address space of the process by mmap.
//
// Both techniques require the call to mmap. What differs is how the descriptor
// that is an argument to mmap is obtained: by open or shm_open.
//
// ****************************************************************************
#ifndef SHMEM_H_
#define SHMEM_H_

#include <errno.h>      // error checking
#include <fcntl.h>      // O_ constants
#include <semaphore.h>  // semaphore API
#include <stddef.h>     // defines NULL
#include <sys/mman.h>   // shared memory API
#include <sys/stat.h>   // mode constants
#include <unistd.h>     // for close()

#include <iostream>
using namespace std;

template <class T, long count = 1>
class shmem
{
public:

    // ------------------------------------------------------------------------
    shmem(const char* name) :
        pName(name), pShmData(0), mShmFd(0), mCreated(true)
    {
        cout << "START: shmem(\"" << pName << "\", " << count << ") Constructor" << endl<< flush;

        // --------------------------------------------------------------------
        // The two-step process involved with POSIX shared memory requires:
        //      1. calling shm_open, specifying a name argument, to either
        //         create a new shared memory object or to open an existing
        //         shared memory object, followed by
        //      2. calling mmap to map the shared memory into the address space
        //         of the calling process.
        int    flags = O_RDWR|O_CREAT|O_EXCL;
        mode_t mode  = S_IRUSR|S_IWUSR;

        // flag indicating that the shared memory is the same as the data
        // passed in
        bool valid = true;

        // Determine the amount of memory should include the
        // header + the data buffer
        const size_t len = sizeof(shmem_data_t);

        cout << "Shmem_open()... "<< flush;
        mShmFd = shm_open(pName, flags, mode);

        // Check to see if the shared memory has been created yet
        if (mShmFd == -1)
        {
            cout << "failed. ********************* errno: " << errno << endl<< flush;

            // Remove flags (O_EXCL, O_CREAT) and try to open shared memory
            // that already exists
            flags &= ~O_EXCL;
            flags &= ~O_CREAT;

            cout << "Shmem_open (Again)... "<< flush;
            mShmFd = shm_open(pName, flags, mode);

            // Check to see if an error occurred while trying to open
            valid = (mShmFd != -1);

            if (valid)
                        {
                cout << "success!" << endl<< flush;

                            // Indicate that the shared memory already existed
                            mCreated = false;
                        }
            else
                        {
                cout << "failed. ********************* errno: " << errno << endl<< flush;
                        }

        } else
        {
            cout << "success!" << endl << flush;
        }

        cout << "mmap()... "<< flush;
        // The mmap function maps a POSIX shared memory object (T) + Header
        // into the address space of a process.
        pShmData = reinterpret_cast<shmem_data_t*> (
                mmap(NULL, len, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, mShmFd, 0));

        if (pShmData == NULL)
        {
            int error = errno;

            switch (error)
            {
            default:
                // Undefined Error
                cout << "failed: ********************* errno: " << error << endl<< flush;
                break;
            }
        } else
        {
            cout << "success: " << hex << "0x" << pShmData << dec << endl << flush;
        }

        // Check to see if we are the first user to request this shared memory
        // location.
        if (mCreated)
        {
            cout << "CREATED!" << endl;
            cout << "Initial Header Data: Size  " << pShmData->size  << endl;
            cout << "Initial Header Data: Count " << pShmData->len << endl;

            // Initialize the header if we created the SHM
            cout << "sem_init()" << endl<< flush;
            sem_init(&pShmData->mutex,1,1);

            cout << "sem_wait()... " << endl<< flush;
            sem_wait(&pShmData->mutex);

            cout << "Got It!" << endl<< flush;
            pShmData->size  = len;
            pShmData->len = count;

            cout << "release semaphore" << endl<< flush;
            sem_post(&pShmData->mutex);
            cout << "Initialization complete" << endl<< flush;

            cout << "Header Data: Size  " << pShmData->size  << endl;
            cout << "Header Data: Count " << pShmData->len << endl;

        } else if (valid)
        {

            cout << "Validating Shared Memory... " ;

            // Validate the Shared Memory that was acquired
            valid &= (pShmData->size == len);
            valid &= (pShmData->len == count);

            if (valid)
                cout << "success!" << endl<< flush;
            else
                cout << "failed. ********************* " << endl<< flush;

            cout << "Header Data: Size  " << pShmData->size  << endl;
            cout << "Header Data: Count " << pShmData->len << endl;


        }
else
{
shm_unlink(pName);
exit(1);
}

                // FIXME: What should we do if we aren't valid?!
        cout << "END: Shmem Constructor" << endl<< flush;

    }

    // ------------------------------------------------------------------------
    // Copy Constructor - Increment Use count for Shared Memory.
    shmem(const shmem& that) :
        pName(that.pName), pShmData(0), mShmFd(0)
    {
        cout << "START: shmem Copy Constructor" << endl << flush;

        // --------------------------------------------------------------------
        // The two-step process involved with POSIX shared memory requires:
        //      1. calling shm_open, specifying a name argument, to either
        //         create a new shared memory object or to open an existing
        //         shared memory object, followed by
        //      2. calling mmap to map the shared memory into the address space
        //         of the calling process.
        int    flags = O_RDWR;
        mode_t mode  = S_IRUSR|S_IWUSR;

        // flag indicating that the we allocated valid shared memory is the
        // same as the data passed in
        bool valid = true;

        // Determine the amount of memory should include the
        // header + the data buffer
        const size_t len = sizeof(shmem_data_t);

        mShmFd = shm_open(pName, flags, mode);

        // Check to see if an error occurred while trying to open
        valid = (mShmFd != -1);

        // The mmap function maps a POSIX shared memory object (T) + Header
        // into the address space of a process.
        pShmData = mmap(NULL, that.mShmFd->size, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_ANONYMOUS, mShmFd, 0);

        cout << "close()... "<< flush;
        // The close() function will deallocate the file descriptor.
        // All outstanding record locks owned by the process on the file
        // associated with the file descriptor will be removed (that is,
        // unlocked).
        //
        // If a shared memory object remains referenced at the last close (that is, a
        // process has it mapped), then the entire contents of the memory object persist
        // until the memory object becomes unreferenced. If this is the last close of a
        // memory object and the close results in the memory object becoming
        // unreferenced, and the memory object has been unlinked, then the memory object
        // will be removed.
        close(mShmFd);
        cout << "success." << endl<< flush;


        cout << "END: shmem Copy Constructor" << endl << flush;

    }


    // ------------------------------------------------------------------------
    virtual ~shmem()
    {

        cout << "START: ~shmem() Destructor" << endl<< flush;

                if (mCreated)
                {
        cout << "shm_unlink( \"" << pName << "\")... "<< flush;

        // The shm_unlink function removes the name of a shared memory object.
        // As with all the other unlink functions, unlinking a name has no
        // effect on existing references to the underlying object, until all
        // references to that object are closed. Unlinking a name just prevents
        // any subsequent call to open, mq_open, or sem_open from succeeding.
        if(shm_unlink(pName) == -1)
        {
            int error = errno;

            switch (error)
            {
            case EACCES:
                // Permission is denied to unlink the named shared memory
                // object.
                cout << "Failed: ********************* EACCES " << endl<< flush;
                break;

            case ENAMETOOLONG:
                // The length of the name argument exceeds {PATH_MAX} or a
                // pathname component is longer than {NAME_MAX}.
                cout << "Failed: ********************* ENAMETOOLONG" << endl<< flush;
                break;

            case ENOENT:
                // The named shared memory object does not exist.
                cout << "Failed: ********************* ENOENT" << endl<< flush;
                break;

            default:
                // Undefined Error
                cout << "Failed: ********************* <UNKNOWN> errno: " << error << endl<< flush;
                break;
            }
        } else
        {
            cout << "Success!" << endl<< flush;
        }

                }

        cout << "close()... " << flush;
        // The close() function will deallocate the file descriptor.
        // All outstanding record locks owned by the process on the file
        // associated with the file descriptor will be removed (that is,
        // unlocked).
        //
        // If a shared memory object remains referenced at the last close (that is, a
        // process has it mapped), then the entire contents of the memory object persist
        // until the memory object becomes unreferenced. If this is the last close of a
        // memory object and the close results in the memory object becoming
        // unreferenced, and the memory object has been unlinked, then the memory object
        // will be removed.
        close(mShmFd);
        cout << "success." << endl << flush;



        cout << "END: ~shmem() Destructor" << endl<< flush;
    }


    // ------------------------------------------------------------------------
    // Returns address only to the indexed object in shared memory
    T* Obj_Addr(uint32_t n = 0)
    {
        cout << "shmem.Obj_Addr()" << endl << flush;
        return &pShmData->buf[n];
    }

    // ------------------------------------------------------------------------
    // sync...
    void Sync()
    {
                cout << "shmem.Sync()... ";
               if (msync(pShmData, sizeof(shmem_data_t), MS_SYNC) == -1)
               {
                   cout << "failed: ********************* errno: " << errno << endl<< flush;
               } else
               {
                   cout << "success. " << endl << flush;
               }        
    }

    // ------------------------------------------------------------------------
    // Returns reference only to the indexed object in shared memory
    T& Obj(uint32_t n = 0)
    {
        cout << "shmem.Obj()" << endl << flush;
        return pShmData->buf[n];
    }

    // ------------------------------------------------------------------------
    // Returns reference only to the indexed object in shared memory
    T& operator[] (uint32_t n)
    {
        cout << "Accessing shmem[" << n << "] == " << flush;
        cout << pShmData->buf[n] << "!"  << endl << flush;
        return pShmData->buf[n];
    }


private:
    // ------------------------------------------------------------------------
    // Hide default constructor
    shmem() : pName(0), pShmData(0), mShmFd(0)
    {

    }

private:
    struct shmem_data_t
    {
        size_t   size;
        uint32_t len;
        sem_t    mutex;
        T        buf[count];
    };

    const char*   pName;
    shmem_data_t* pShmData;
    int           mShmFd;
        // Flag indicating that we created the shared memory
bool mCreated;


};

#endif /* SHMEM_H_ */

main.cpp

#include <signal.h>
#include <stdlib.h>
#include <string.h>

#include <iostream> // ** FIXME ** DEBUG
using namespace std;

#include "stdint.h"
#include "shmem.h"

bool done = false;

// ----------------------------------------------------------------------------
void my_handler(int s)
{
          cout << "Goodbye! SIG: " << s << endl << flush;
          done = true;
}

// ----------------------------------------------------------------------------
void test_shmem()
{

    cout << endl << endl << "Testing Shmem Template" << endl;
    cout << "-------------------------------------------" << endl;
    shmem<int,13> x("/jco");
    cout << "-------------------------------------------" << endl;
    shmem<int,13> y("/jco");
    cout << "-------------------------------------------" << endl;
    x[5] = 7;
        x.Sync();
    cout << "-------------------------------------------" << endl;
    cout << "X[5] = " << x[5] << endl;
    cout << "-------------------------------------------" << endl;
    cout << "Y[5] = " << y[5] << endl;
    cout << "-------------------------------------------" << endl;
    cout << endl << "*** Testing Complete." << endl << endl;

    sleep(10);

}

// ----------------------------------------------------------------------------
int main()
{
    cout << "MAIN" << endl;

   struct sigaction sigIntHandler;

   sigIntHandler.sa_handler = my_handler;
   sigemptyset(&sigIntHandler.sa_mask);
   sigIntHandler.sa_flags = 0;

   sigaction(SIGINT, &sigIntHandler, NULL);


    test_shmem();

    // RUN
    while(not done)
    {
        sleep(1);
    }

    return 0;
}

console output:

MAIN


Testing Shmem Template
-------------------------------------------
START: shmem("/jco", 13) Constructor
Shmem_open()... success!
mmap()... success: 0x0x7f32113ad000
CREATED!
Initial Header Data: Size  0
Initial Header Data: Count 0
sem_init()
sem_wait()...
Got It!
release semaphore
Initialization complete
Header Data: Size  104
Header Data: Count 13
END: Shmem Constructor
-------------------------------------------
START: shmem("/jco", 13) Constructor
Shmem_open()... failed. ********************* errno: 17
Shmem_open (Again)... success!
mmap()... success: 0x0x7f32113ac000
Validating Shared Memory... failed. *********************
Header Data: Size  0
Header Data: Count 0
END: Shmem Constructor
-------------------------------------------
Accessing shmem[5] == 0!
shmem.Sync()... success.
-------------------------------------------
Accessing shmem[5] == 7!
X[5] = 7
-------------------------------------------
Accessing shmem[5] == 0!
Y[5] = 0
-------------------------------------------

*** Testing Complete.

START: ~shmem() Destructor
close()... success.
END: ~shmem() Destructor
START: ~shmem() Destructor
shm_unlink( "/jco")... Success!
close()... success.
END: ~shmem() Destructor
Goodbye! SIG: 2

Answer 1

Edit: my first answer completely missed the mark. So I feel obligated to contribute something useful.

Both Petesh and BЈовић gave correct answers. First, you should not be using MAP_ANONYMOUS . Second, you should realize that the (virtual) address that you get back from mmap will not be the same as the first one. When you call mmap twice, you are creating two separate mappings to the same shared memory. But you can use either of the two addresses and they will be pointing at the same piece of shared memory.

I wrote this small program to demonstrate. It basically does what your program does, and shows that even though the two mmap calls return two different addresses, both addresses are reading and writing to the same shared memory.

#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <unistd.h>

#define TEMP_FILENAME   "shm.tmp"

int main(void)
{
    int fd1 = shm_open(TEMP_FILENAME, O_CREAT | O_RDWR, 0777);
    int fd2 = shm_open(TEMP_FILENAME, O_RDWR, 0777);
    int *p1, *p2;
    int buf[1024] = {0x12345678};

    // Write initial contents to shared memory.
    write(fd1, buf, 4096);
    p1 = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd1, 0);
    p2 = mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, fd2, 0);
    printf("fd1 = %d, p1 = %p\n", fd1, p1);
    printf("fd2 = %d, p2 = %p\n", fd2, p2);
    printf("p1[0] = 0x%08x, p2[0] = 0x%08x\n", p1[0], p2[0]);
    p1[0] = 0xdeadbeef;
    printf("p1[0] = 0x%08x, p2[0] = 0x%08x\n", p1[0], p2[0]);
    close(fd2);
    close(fd1);
    shm_unlink(TEMP_FILENAME);
    return 0;
}

And the output:

fd1 = 3, p1 = 0x7f2b3d434000
fd2 = 4, p2 = 0x7f2b3d433000
p1[0] = 0x12345678, p2[0] = 0x12345678
p1[0] = 0xdeadbeef, p2[0] = 0xdeadbeef

Answer 2

From mmap() man pages :

mmap() creates a new mapping in the virtual address space of the calling process.

So, what you get is a virtual address, matching to the same physical address. Since you are mmap()-ing the same memory region in 2 different processes, you are going to get different virtual address.

You should read more about linux memory management (for example here ).

Answer 3

You've got an incorrect argument to mmap - the MAP_ANONYMOUS option.

What that does is explicitly ignore the file descriptor from shm_open and instead creates a private mapping for the process.

The manpage for mmap for MAP_ANONYMOUS indicates:

The mapping is not backed by any file; its contents are initialized to zero. The fd and offset arguments are ignored.