Manually call a rust dynamic library

Question

I'm currently playing around with DynamicLibrary .

The code of my dynamic library (compiled with rustc --crate-type dylib dylib.rs ):

// dylib.rs
#[no_mangle]
pub fn minicall() -> u8 {
    3u8
}

And the code to call it:

// caller.rs
use std::dynamic_lib::DynamicLibrary;

fn main() {
    let mut v = Vec::new();

    DynamicLibrary::prepend_search_path(&::std::os::getcwd());

    match DynamicLibrary::open(Some("./libdylib.so")) {
        Err(e) => panic!("ERROR: {}", e),
        Ok(lib) => {
            println!("Unsafe bloc !");
            let func = unsafe {
                match lib.symbol::< fn() -> u8 >("minicall") {
                        Err(e) => { panic!("ERROR: {}", e) },
                        Ok(f) => { *f },
                }
            };
            println!("call func !");
            let new_value = func();

            println!("extend vec !");
            v.push(new_value);
        }
    }

    println!("v is: {}", v);
}

I have this output :

~> ./caller 
Unsafe bloc !
call func !
Illegal instruction

And here I'm quite lost. What am I doing wrong ?

Answer 1

The problem here is how the symbol function works. It has signature:

unsafe fn symbol<T>(&self, symbol: &str) -> Result<*mut T, String>

A loaded library is basically a big array in memory with certain addresses labelled with a name (the symbol names). Querying for a symbol looks up the address and returns a pointer straight to it. A function in a library is a long sequence of instructions, so querying for a function's name returns a (function) pointer directly to the start. This can then be called as a normal function pointer. The Rust DynamicLibrary API is returning this pointer, that is, *mut T points directly to the chunk of memory in the dynamic library (which is supposedly/hopefully of type T ).

The type fn(...) -> ... is a function pointer itself, that is, it is 8 bytes (or 4 bytes) storing the address of the start of the function it represents. Hence, calling lib.symbol::< fn() -> u8 >("minicall") is saying "find me the address of the thing called minicall (which is a pointer to a function )", it is not saying "find me the address of the thing called minicall (which is a function)". The return value of *mut (fn() -> u8) is then doubly-indirect, and dereferencing it to call it is interpreting the first 8 (or 4) bytes of the function code as a pointer (ie random machine instructions/function prelude), it is not executing them.

(Side-note: it would probably work if you had #[no_mangle] pub static minicall: fn() -> u8 = the_real_minicall; in your library, but you probably don't want this.)

The call to lib.symbol::<T>("minicall") is returning the exact function pointer we want (that is, it is returning a pointer to the start of the code of minicall ), so it just becomes a question of expressing this to the compiler. Unfortunately, there is currently no type T that makes *mut T a function pointer, so one must first set T = u8 (ie lib.symbol::<u8>("minicall") ) and then cast the return value to the appropriate function pointer type via transmute::<_, fn() -> u8>(pointer) .

(I'm answering this even after the other answer was accepted because I don't think it explained the cause very well, just gave the solution.)

---

Last thing, this isn't a problem in this case, but it trips people a lot: the Rust ABI (the calling convention used for functions of type fn(...) -> ... ) is not the same as the C ABI, so functions loaded from C dynamic libraries should be given type extern "C" fn(...) -> ... , not fn(...) -> ... .

Answer 2

I think the problem stems from the fact that you are casting between incompatible types. Specifically, the dereference *f is going to point to the wrong place. I looked in the Rust code to see how the library is supposed to be used and found an example in src/librustc/plugin/load.rs . I adapted that code to your example:

let func = unsafe {
    // Let this return a `*mut u8`, a very generic pointer
    match lib.symbol("minicall") {
        Err(e) => { fail!("ERROR: {}", e) },
        // And then cast that pointer a function
        Ok(f) => { std::mem::transmute::<*mut u8, fn() -> u8>(f) },
    }
};
println!("call func !");
let new_value = func();

The output:

$ ./caller
Unsafe bloc !
call func !
extend vec !
v is: [3]

Answer 3

Since this question/answer, the std::dynamic_lib API seems to have gone away. As of this writing it looks like libloading is the most popular way of dynamically loading libraries on crates.io.