Using ftrace on simple program, Inline Assembly __asm__(“leave”) resulting in seg fault

Question

I am reading through this book on learning Linux Binary Analysis. In the book the author presents ftrace, which he has on his github and demonstrates how to use it. He provides a small bit of code with which to test the ftrace.

When running the ftrace on it, nothing happens. If I run the executable on its own I just get a seg fault. I am compiling it like so: gcc -nostdlib test.c -o test

int foo(void) {  
}

_start()
{
    foo();
    __asm__("leave");
}

The expected results demonstrate the ftrace tracing the function calls through the execution.

https://github.com/elfmaster/ftrace

I guess the question is, am I missing something completely, doing something wrong, is the text outdated or what would be the correct approach to this? I apologize if this is a dumb question, I was just going off the text. I also tried this with a 32bit distro on a VM and nothing changed, but just tried it because some of the author's examples are on 32bit. Thank you.

When I run his ftrace with a program that doesn't cause a seg fault I get 当我使用不会引起段错误的程序运行他的ftrace时，我得到

pid_read() failed: Input/output error <0x1>

Answer 1

Call _exit(0); or exit_group(0) at the end of _start . (Link with gcc -static -nostartfiles instead of -nostdlib so you can call the libc system call wrapper functions; they should work even though glibc init functions haven't been run so malloc or printf would crash ).

Or make an exit_group(0) system-call manually with inline asm. On x86-64 Linux:
asm("mov $231, %eax; xor %edi,%edi; syscall");

See also How Get arguments value using inline assembly in C without Glibc? for more about writing a hacky x86-64 _start to run your own C function as the first thing in your process. (But most of that answer is about hacking the calling convention to access argc / argv, which is nasty and I don't recommend it.) Matteo's answer on that question has a whole minimal _start written in asm that calls a normal C main function.

---

The book's code is just plain broken for 2 reasons . (I don't know how it could have ever worked on i386 or x86-64. Seems super weird to me. Are you sure it wasn't just supposed to crash, but you look at what it does before that happens?)

1. _start is not a function in Linux; you (or compiler-generated code) can't ret from it . You need to make an _exit system call. There is no return address on the stack ¹ .

   Where a function would have its return address, the ELF entry point _start has argc , as specified in the ABI docs. (x86-64 System V or i386 System V depending on whether you build a 64-bit or gcc -m32 32-bit executable.)

2. Inserting leave (which does mov %ebp, %esp / pop %ebp or the RBP / RSP equivalent) into the compiler-generated code makes no sense here. It's sort of like an extra pop , but breaks the compiler's EBP / RBP so if it happens to choose leave instead of pop %rbp for its own prologue then the compiler generated code will fault. (RBP on entry to _start is 0 in a statically-linked executable. Or holding whatever the dynamic linker left in RBP before jumping to _start in a PIE executable.)

   But ultimately, GCC will compile _start as a normal function, thus eventually running a ret instruction. There is no valid / useful return address anywhere, so there's no way ret can work at all.

   If you compile without optimization (the default), gcc will default to -fno-omit-frame-pointer , so its function prologue will have set up EBP or RBP as a frame pointer making it possible for leave itself to not fault. If you compiled with optimization ( -O1 and higher enables -fomit-frame-pointer ), gcc wouldn't be messing with RBP, and it would be zero when you ran leave , thus directly causing a segfault. (Because it does RSP=RBP and then uses the new RSP as the stack pointer for pop %rbp .)

Anyway, if it doesn't fault, that will leave the stack pointer pointing to argc again, before the compiler-generated pop %rbp as part of the normal function epilogue. So the compiler-generated ret will try to return to argv[0] . Since the stack is non-executable by default, that will segfault. (And it's pointing to ASCII characters, which probably don't decode as useful x86-64 machine code.)

You could have found this out yourself by single-stepping the asm with GDB. ( layout reg and use stepi aka si ).

In general you messing with the stack pointer and other registers behind the compiler's back will typically just make things crash. And if there had been a return address higher up on the stack, pop %rcx would make a lot more sense than leave .

---

Footnote 1:

There's not even any machine code anywhere in the address space of your process that a useful return address could point to to make such a system call, unless you inject some machine code as an arg or environment variable.

You linked with -nostdlib so there's no libc linked. If you did link libc dynamically but still wrote your own _start (eg with gcc -nostartfiles instead of the full -nostdlib ), ASLR would mean the libc _exit function was at some runtime-variable address.

If you statically linked libc ( gcc -nostartfiles -static ), the code for _exit() wouldn't be copied into your executable unless you actually referenced it, which this code doesn't. But you still need to get it called somehow; there's no return address pointing to it.