C程序的主要功能是否曾经回收过堆栈？

Question

I am working through the OverTheWire wargames and one of my exploits overwrites the return address of main with the address of system . I have then used the fact that at the point main returns, esp is still pointing at one of my local variables and hence I can fill it with the command I want system to run (eg sh;# ).

My confusion comes from that I thought functions in C reclaim the stack before returning and hence at the point the return address is called the stack pointer would be pointing at the return address rather than at the local variables. However, my exploit works so it seems that my stack pointer is pointing at the local variables when the return address is called.

The main thing I have noticed about this particular challenge compared to others is that it calls exit(0) at the end, instead of just ending, so the assembly doesn't end with leave , which may be the reason for this behaviour.

I haven't included the actual code since it's quite long and I was hoping there was a general explanation for what I am seeing, but please let me know if the assembly would be useful.

Answer 1

#include <stdio.h>
int main ( void )
{
    printf("hello\n");
    return(0);
}

the interesting relevant parts.

0000000000400430 <main>:
  400430:   48 83 ec 08             sub    $0x8,%rsp
  400434:   bf d4 05 40 00          mov    $0x4005d4,%edi
  400439:   e8 c2 ff ff ff          callq  400400 <puts@plt>
  40043e:   31 c0                   xor    %eax,%eax
  400440:   48 83 c4 08             add    $0x8,%rsp
  400444:   c3                      retq   
  400445:   66 2e 0f 1f 84 00 00    nopw   %cs:0x0(%rax,%rax,1)
  40044c:   00 00 00 
  40044f:   90                      nop



0000000000400450 <_start>:
  400450:   31 ed                   xor    %ebp,%ebp
  400452:   49 89 d1                mov    %rdx,%r9
  400455:   5e                      pop    %rsi
  400456:   48 89 e2                mov    %rsp,%rdx
  400459:   48 83 e4 f0             and    $0xfffffffffffffff0,%rsp
  40045d:   50                      push   %rax
  40045e:   54                      push   %rsp
  40045f:   49 c7 c0 c0 05 40 00    mov    $0x4005c0,%r8
  400466:   48 c7 c1 50 05 40 00    mov    $0x400550,%rcx
  40046d:   48 c7 c7 30 04 40 00    mov    $0x400430,%rdi
  400474:   e8 97 ff ff ff          callq  400410 <__libc_start_main@plt>
  400479:   f4                      hlt    
  40047a:   66 0f 1f 44 00 00       nopw   0x0(%rax,%rax,1)

For the most part there is nothing special about main nor printf, etc these are just functions that conform to the calling convention. As re-asked SO questions will show sometimes the compiler will add extra stack or other calls when it sees a main() that it doesnt otherwise. but still it is a function that needs to conform to the calling convention. As seen in this case where the stack pointer is put back where it was found.

Before an operating system (Linux, Windows, MacOS, etc) can even think about running a program it needs to allocate some space for that program and tag that memory for that program in some way depending on the features of the processor and the OS, etc. Then you load the program from whatever media, and launch it at the binary file specified and/or well known entry point. A clean exit of the program will cause the operating system to free that memory, which the .text, .data, .bss and stack are the trivial/obvious ones that just go away as their memory just goes away. Other items that may have been allocated and associated with this program, open files, runtime allocated (not stack) memory, etc can/should also be freed, depends on the design of the os and/or the C library as to how that happens.

In the above case we see the bootstrap calls main and main returns then hlt is hit which this is an application not kernel code so that should cause a trap that causes the OS to clean up. An explicit exit() should be no different than a printf() or puts() or fopen() or any other function that ultimately makes one or more syscalls to the operating system. All that you can possibly find for these types of operating systems (Linux, Windows, MacOS) is the syscall. The release of memory happens outside the program as the program does not have control over it, would be a chicken and egg problem, program frees mmu tables that is using to free mmu tables...

compile and disassemble the object for main rather than the whole program

0000000000000000 <main>:
   0:   48 83 ec 08             sub    $0x8,%rsp
   4:   bf 00 00 00 00          mov    $0x0,%edi
   9:   e8 00 00 00 00          callq  e <main+0xe>
   e:   31 c0                   xor    %eax,%eax
  10:   48 83 c4 08             add    $0x8,%rsp
  14:   c3                      retq 

no surprise there same as before, all we needed to see to understand that the stack was cleaned up before return. and that main is not special:

#include <stdio.h>
int notmain ( void )
{
    printf("hello\n");
    return(0);
}

0000000000000000 <notmain>:
   0:   48 83 ec 08             sub    $0x8,%rsp
   4:   bf 00 00 00 00          mov    $0x0,%edi
   9:   e8 00 00 00 00          callq  e <notmain+0xe>
   e:   31 c0                   xor    %eax,%eax
  10:   48 83 c4 08             add    $0x8,%rsp
  14:   c3                      retq   

Now if you are asking if there is an exit() within main then sure it wont hit the return point in main so the stack pointer is offset by whatever amount. but if main calls some function and that function calls some function then that function calls exit() then the stack pointer is left at the stack frame point of function number two plus whatever the call (this is an x86) plus the exit() stack frame adds to it. You cannot simply assume that when exit() is called, if it is called, what the stack pointer is pointing at. You would have to examine the disassembly around that call to exit() plus the exit() code and anything it calls, to figure this out.