Why does dlsym produce different results in cgo than in c?

Question

I have two implementations of the same behavior that I believe should produce the same results but are instead producing different results. When compiled in Go using cgo , I get a different symbol address resolution than when compiled in C. I would like to understand why.

I reduced the problem to a couple of small examples, one in C and one in Go. I tested these in an Ubuntu 18 Docker container running on my Mac laptop.

test.c:

// gcc test.c -D_GNU_SOURCE -ldl
// Output: Real: 0x7fd05559d7d0 Current: 0x7fd05559d7d0

#include <dlfcn.h>
#include <stdio.h>

int main() {
    void * fd = dlopen("libc.so.6", RTLD_LAZY);
    void * real_sym = dlsym(fd, "accept");
    void * curr_sym = dlsym(RTLD_NEXT, "accept");
    printf("Real: %p Current: %p\n", real_sym, curr_sym);
    return 0;
}

test.go:

// go build test.go
// Output: Real: 0x7f264583b7d0 Current: 0x7f2645b1b690
package main

// #cgo CFLAGS: -D_GNU_SOURCE
// #cgo LDFLAGS: -ldl
// #include <dlfcn.h>
import "C"
import "fmt"

func main() {
    fp := C.dlopen(C.CString("libc.so.6"), C.RTLD_LAZY)
    real_sym := C.dlsym(fp, C.CString("accept"))
    curr_sym := C.dlsym(C.RTLD_NEXT, C.CString("accept"))
    fmt.Printf("Real: %p Current: %p\n", real_sym, curr_sym)
}

I get the output of Real: 0x7fd05559d7d0 Current: 0x7fd05559d7d0 when test.c gets compiled ( gcc test.c -D_GNU_SOURCE -ldl ). However, when I build test.go I see Real: 0x7f264583b7d0 Current: 0x7f2645b1b690 .

I assume that go is wrapping some symbols itself, but I would like to know exactly what's happening. Thanks!

---

A couple of extra pieces after seeing some of the initial comments. I changed test.c as below and then ran in a loop ( while [ 1 ]; do ./a.out; done ). It's consistently getting equal addresses for me (different each run, though).

// gcc test.c -D_GNU_SOURCE -ldl
// Output: Real: 0x7fd05559d7d0 Current: 0x7fd05559d7d0

#include <dlfcn.h>
#include <stdio.h>

    int main() {
    void * fd = dlopen("libc.so.6", RTLD_LAZY);
    void * real_sym = dlsym(fd, "accept");
    void * curr_sym = dlsym(RTLD_NEXT, "accept");
    if(real_sym != curr_sym) {
        printf("Real: %p Current: %p\n", real_sym, curr_sym);
    }
    return 0;
}

I also tried a modification of the Go code to check if it had to do with how Go called out to C, but that still did not have the addresses match:

// go build dos.go
// Output: Real: 0x7f264583b7d0 Current: 0x7f2645b1b690
package main

// #cgo CFLAGS: -D_GNU_SOURCE
// #cgo LDFLAGS: -ldl
// #include <dlfcn.h>
// #include <stdio.h>
// int doit() {
//     void * fd = dlopen("libc.so.6", RTLD_LAZY);
//     void * real_sym = dlsym(fd, "accept");
//     void * curr_sym = dlsym(RTLD_NEXT, "accept");
//     printf("Real: %p Current: %p\n", real_sym, curr_sym);
//     return 0;
// }
import "C"

func main() {
    C.doit()
}

Another point is that I get the two addresses to match in both the C and Go code if I look for the malloc symbol instead of accept .

Answer 1

The symbols aren't loaded into fixed addresses in memory; they go wherever the loader decides to put them.

This is the output of me running your C program multiple times on my machine.

govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7f4b5f3127d0 Current: 0x7f4b5f26ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7f45727127d0 Current: 0x7f457266ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7fc3373127d0 Current: 0x7fc33726ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7f0e555127d0 Current: 0x7f0e5546ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7f2fdd9127d0 Current: 0x7f2fdd86ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7fec7db127d0 Current: 0x7fec7da6ee30
govind@Govind-PC:/mnt/c/Temp$ ./dlst
Real: 0x7f07de1127d0 Current: 0x7f07de06ee30
govind@Govind-PC:/mnt/c/Temp$

See also:

Address Space Layout Randomization

Answer 2

The reason is that Go links against libpthread, but your C program doesn't. If I add -lpthread to the gcc arguments, it prints different pointers too. So, libpthread defines its own accept and overrides the libc one (which kind of makes sense).

The way I figured that out is that I inserted a sleep into both programs and then rummaged through /proc/$pid/maps to see what the returned pointers reference. That showed that in Go's case, the "current" pointer resides in libpthread. The "real" pointer always references libc.