Memory map an array at a fixed location

Question

To verify a program, I need to generate a deterministic address trace. So I am using mmap to allocate an array at a fixed location in virtual memory.

This is what I have:

#define PG_SIZE 4096
#define PG_START(_v) ((_v) & ~(unsigned long)(PG_SIZE-1))

// has to be aligned to page. otherwise it will be done for you.
#define ARRAY_ONE_BASE PG_START(0xffeffd000)
#define ARRAY_ONE_SIZE (4096 * 2 * sizeof(int))


unsigned long * allocateArray(unsigned long addr, size_t size) {

    int stack_prot =  PROT_READ | PROT_WRITE;
    int flags = MAP_SHARED | MAP_FIXED | MAP_ANONYMOUS;
    unsigned long *m_map;

    m_map = mmap((caddr_t)PG_START(addr), size, stack_prot, flags, -1, 0);

    // fail if mmap faield
    if (m_map == MAP_FAILED) {
        perror("mmap failed");
        abort();
    }

    printf("Base address of allocated variable: %li\n", m_map);
    assert((void *)m_map == (void *)addr);

    return m_map;
}


int main(int argc, char *argv[]) {    
    unsigned long *addrOne    = allocateArray(ARRAY_ONE_BASE, ARRAY_ONE_SIZE);
}

If I compile and run above code it works fine. However, when I try to run valgrind tool lackey , it gives a segmentation fault.

Commands:

gcc program.c
valgrind --read-var-info=yes --tool=lackey a.out

Output:

==28528== Lackey, an example Valgrind tool
==28528== Copyright (C) 2002-2013, and GNU GPL'd, by Nicholas Nethercote.
==28528== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==28528== Command: a.out
==28528==
==28528==
==28528== Process terminating with default action of signal 11 (SIGSEGV)
==28528==  Bad permissions for mapped region at address 0x0
==28528==    at 0x0: ???
==28528==
==28528== Counted 1 call to main()
==28528==
==28528== Jccs:
==28528==   total:         22,338
==28528==   taken:         9,947 ( 44%)
==28528==
==28528== Executed:
==28528==   SBs entered:   21,937
==28528==   SBs completed: 14,439
==28528==   guest instrs:  120,810
==28528==   IRStmts:       723,993
==28528==
==28528== Ratios:
==28528==   guest instrs : SB entered  = 55 : 10
==28528==        IRStmts : SB entered  = 330 : 10
==28528==        IRStmts : guest instr = 59 : 10
==28528==
==28528== Exit code:       0
Segmentation fault (core dumped)

I have tried different combinations of protection flags on mmap but have had no luck. I'd appreciate any help.

Answer 1

There's no guarantee that mmap() with MAP_FIXED will always work.

http://pubs.opengroup.org/onlinepubs/007908799/xsh/mmap.html has:

When MAP_FIXED is set in the flags argument, the implementation is informed that the value of pa must be addr, exactly. If MAP_FIXED is set, mmap() may return MAP_FAILED and set errno to [EINVAL].

and Linux manpage has:

MAP_FIXED

Don't interpret addr as a hint: place the mapping at exactly that address. addr must be a multiple of the page size. If the memory region specified by addr and len overlaps pages of any existing mapping(s), then the overlapped part of the existing mapping(s) will be discarded. If the specified address cannot be used, mmap() will fail. Because requiring a fixed address for a mapping is less portable, the use of this option is discouraged.

For example, I had similar results running your program on Ubuntu 14.04 x64 , but it failed on my Ubuntu 14.04 x86 box (which is natural consider the address itself):

$ ./a.out
mmap failed: Cannot allocate memory
Aborted

Moreover, Valgrind puts more restrictions on MAP_FIXED:

(Line 076)

• Try to place the client's fixed and hinted mappings at the requested addresses. Fixed mappings are allowed anywhere except in areas reserved by Valgrind ; the client can trash its own mappings if it wants.

I then changed ARRAY_ONE_BASE in your program from 0xffeffd000 to 0x10ffeffd000, and it worked:

$ valgrind --read-var-info=yes --tool=lackey ./a.out
==6319== Lackey, an example Valgrind tool
==6319== Copyright (C) 2002-2013, and GNU GPL'd, by Nicholas Nethercote.
==6319== Using Valgrind-3.10.0.SVN and LibVEX; rerun with -h for copyright info
==6319== Command: ./a.out
==6319== 
Base address of allocated variable: 0x10ffeffd000
==6319== 
==6319== Counted 1 call to main()
==6319== 
==6319== Jccs:
==6319==   total:         20,945
==6319==   taken:         9,423 ( 44%)
==6319== 
==6319== Executed:
==6319==   SBs entered:   19,675
==6319==   SBs completed: 12,655
==6319==   guest instrs:  105,453
==6319==   IRStmts:       614,550
==6319== 
==6319== Ratios:
==6319==   guest instrs : SB entered  = 53 : 10
==6319==        IRStmts : SB entered  = 312 : 10
==6319==        IRStmts : guest instr = 58 : 10
==6319== 
==6319== Exit code:       0

I'm no expert to Valgrind or Linux address layout, but it looks like your handpicked address 0xffeffd000 happened to hit some Valgrind reserved area.