Confused about mmap() return - two different pointers?

Question

Currently trying to understand how memory mapping works in Linux (or in general, really), and I'm following with this one example of Shared Memory in POSIX systems from Operating System Concepts. The two files are as follows:

Producer file

// This is the producer file for the Shared memory object

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>

#include <sys/mman.h>

int main()
{
// ---------------- PRODUCER ESTABLISHES SHARED MEMORY OBJECT AND WRITES TO IT ----------------

    // Specifying the size in bytes of the shared memory object
    const int SIZE = 4096;
    
    // Name of the shared memory space
    const char *name = "OS";
    
    // The actual strings to write to shared memory
    const char *message_0 = "Hello";
    const char *message_1 = "World!";
    
    // Shared memory file descriptor will be stored in this
    int fd;
    
    // Pointer to the shared memory object will be stored in this
    char *ptr;
    
    // Checking error
    int errnum;
    
    // Create a shared memory object. This opens (establishes a connection to) a shared memory object.
    fd = shm_open(name, O_CREAT | O_RDWR,0666);
    
    if (fd == -1)
    {
        errnum = errno;
        fprintf(stdout, "Value of errno: %d\n", errno);
        perror("Error printed by perror");
        fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
        return 1;
    }
    
    // Configure the size of the shared-memory object to be 4096 bytes
    ftruncate(fd, SIZE);
    
    // Memory map the shared memory object
    ptr = (char *) mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    
    printf("Ptr is: %p\n", ptr);
    
    if (ptr == MAP_FAILED)
    {
        errnum = errno;
        fprintf(stdout, "Value of errno in ptr: %d\n", errno);
        perror("Error printed by perror");
        fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
        return 1;
    }
    
    // Write to the shared memory object
    sprintf(ptr, "%s", message_0);
    ptr += strlen(message_0);
    sprintf(ptr, "%s", message_1);
    ptr += strlen(message_1);
    
    return 0;
}

Consumer file

// This is the consumer file for the Shared memory object, in which it reads what is in the memory object OS

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <sys/shm.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <errno.h>

#include <sys/mman.h>

int main()
{
    // Size in bytes of shared memory object
    const int SIZE = 4096;
    
    // Name of the shared memory space
    const char *name = "OS";
    
    // Shared memory file descriptor will be stored in this
    int fd;
    
    // Pointer to the shared memory object will be stored in this
    char *ptr;
    
    // Checking error
    int errnum;
    
    // Open the shared memory object
    fd = shm_open(name, O_RDWR, 0666);
    
    // If error in shm_open()
    if (fd == -1)
    {
        errnum = errno;
        fprintf(stdout, "Value of errno: %d\n", errno);
        perror("Error printed by perror");
        fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
        return 1;
    }
    
    
    // Memory-map the shared memory object
    ptr = (char *) mmap(0, SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    
    printf("Ptr is: %p\n", ptr);
    
    if (ptr == MAP_FAILED)
    {
        errnum = errno;
        fprintf(stdout, "Value of errno in ptr: %d\n", errno);
        perror("Error printed by perror");
        fprintf(stdout, "Error opening file: %s\n", strerror(errnum));
        return 1;
    }
    
    // Read from the shared memory object
    printf("%s\n", (char *) ptr);
            
    // Remove the shared memory object (delete it)
    shm_unlink(name);
    
    return 0;
}

When I print the pointer to the shared memory object ( printf("Ptr value: %p\n, ptr) ), I get two different values for the consumer and producer files. Why does this happen?

As I understand, the pointer ptr points at the shared memory object, which is in the physical memory. This physical memory is just shared amongst two processes, by mapping it onto their address space. However, this would require that the pointer point to the same address in physical memory, no? Or is it pointing to the virtual memory (ie the address space of the processes)? If so, does that itself point to the physical memory?

Thanks!

Answer 1

Each process has its own virtual memory address space. The mappings from virtual memory to physical memory are, in general, different for each process. While the shared memory may be in physical memory at address 0x1000, the producer process may have it mapped into virtual memory at address 0x7000, and the consumer process may have it mapped into virtual memory at address 0x4000.

Further, if the shared memory is swapped out of memory for some reason, the system could later reload it to a different physical address, say 0x13000, and update the mappings in the processes so that it appears at the same addresses as before in each of the producer and consumer processes.

Answer 2

Producer and consumer are two different processes, so they have their own virtual memory space each. When you attach the shared segment, you specify the kernel that you have no preference on where (in your unallocated virtual address space) to put it, so you normally get different pointers because most probably both processes have different memory maps. Just think on an scenario where one process has allocated the memory map at, say, address A, and the other process has that address A occupied by some other thing it has allocated (eg dynamic memory for the heap, or a different shared library) It is clear that the kernel cannot allocate two different things in the same (virtual) address range (or even overlap some other mapping), so it uses a differen place, and returns a different pointer.

Just don't worry about it, as both virtual addresses map to the same phisical address (this is sure, as it is a shared memory segment), and the pointers finally end pointing to the same thing. The kernel is normally not aware of where it has placed that same segment in a different process, it doesn't need to do. This means that it is highly improbable that you get both pointers equal, contrary to what you though.

As none of these processes knows about the virtual address space of the other, there's no conflict, but never pass the address to the other process to use, because an address in a virtual address space has absolutely no meaning in the virtual address space of another, different process.

A process has normally no means (and no possibility) to know how the kernel assings the memory and builds the map of virtual to physicall addresses. Even for itself (the kernel also runs in it's virtual memory space, and only deals with the tables that map ---these tables are in the virtual address space of the kernel, but not in the virtual address space of any user process, so this is the reason that makes the kernel capable of changing the mappings but it is not possible for other processes)

It is impossible for a process to know where in actual memory a virtual address points to. That is possible for some administrator processes through a device (and a special device is needed for this) /dev/mem . But if you try to look in there without the actual mapping you'll get a mess of pages some belonging to a process, some to another, with no apparent structure, or you cannot also know what does it mean the contents of those pages.

There's another device /dev/kmem that maps to the kernel virtual address space, and allows a process (eg a debugger) to get access to the mapping table there.... but this is a very dangerous bend, as you can easily crash your system by tweaking there. Think that you cannot normally stop the kernel (this stops the system) or if you can do, you will stop essential parts of the system that should be running.