Smashing the stack not working

Question

I have gone through the walkthrough about smashing the stack. Both the one http://insecure.org/stf/smashstack.html here and one I found on here Trying to smash the stack . I understand what is suppose to be happening, but I can't get it to work properly.

This is just like the other scenarios. I need to skip x=1 and print 0 as the value of x.

I compile with:

gcc file.c

The original code :

void function(){
    char buffer[8];
}

void main(){
    int x;
    x = 0;
    function();
    x = 1;
    printf("%d\n", x);
}

When I run

objdump -dS a.out

I get

0000000000400530 <function>:
  400530:       55                      push   %rbp
  400531:       48 89 e5                mov    %rsp,%rbp
  400534:       5d                      pop    %rbp
  400535:       c3                      retq

0000000000400536 <main>:
  400536:       55                      push   %rbp
  400537:       48 89 e5                mov    %rsp,%rbp
  40053a:       48 83 ec 20             sub    $0x20,%rsp
  40053e:       89 7d ec                mov    %edi,-0x14(%rbp)
  400541:       48 89 75 e0             mov    %rsi,-0x20(%rbp)
  400545:       c7 45 fc 00 00 00 00    movl   $0x0,-0x4(%rbp)
  40054c:       b8 00 00 00 00          mov    $0x0,%eax
  400551:       e8 da ff ff ff          callq  400530 <function>
  400556:       c7 45 fc 01 00 00 00    movl   $0x1,-0x4(%rbp)
  40055d:       8b 45 fc                mov    -0x4(%rbp),%eax
  400560:       89 c6                   mov    %eax,%esi
  400562:       bf 10 06 40 00          mov    $0x400610,%edi
  400567:       b8 00 00 00 00          mov    $0x0,%eax
  40056c:       e8 9f fe ff ff          callq  400410 <printf@plt>
  400571:       c9                      leaveq
  400572:       c3                      retq
  400573:       66 2e 0f 1f 84 00 00    nopw   %cs:0x0(%rax,%rax,1)
  40057a:       00 00 00
  40057d:       0f 1f 00                nopl   (%rax)

In the function I need to figure out how many bytes the return address is beyond the start of the buffer. I am not sure about this value. But since there are 6 bytes from the beginnig of the function to the return; would I add 7 bytes to the buffer?

Then I need to skip the instruction x=1; And since that instruction is 7 bytes long. Would I add 7 to return pointer?

Something like this?

void function(){
    char buffer[8];
    int *ret = buffer + 7;
    (*ret) += 7;
}

void main(){
    int x;
    x = 0;
    function();
    x = 1;
    printf("%d\n", x);
}

This throws the warning:

warning: initialization from incompatible pointer type [enabled by default]
  int *ret = buffer1 + 5;
         ^

And the output is 1. What am I doing wrong? And can you explain how to do it right and why it is the correct way?

Thank you.

Answer 1

We know that automatic variables are created on the stack - so taking the address of an automatic variable yields a pointer into the stack. When you call a void function, its return address is pushed onto the stack and the size of that address depends on your platform (4 or 8 bytes normally). So if you pass the address of an automatic variable to a function and then write over the memory before that address, you will damage the return address and smash the stack. Here is an example:

#include <stdlib.h>
#include <stdio.h>

static void f(int *p)
{
    p[0] = 0x30303030;
    p[1] = 0x31313131;
    *(p - 1) = 0x35353535;
    *(p - 2) = 0x36363636;
}

int main()
{
    int a = 0x41424344;
    int b = 0x45464748;
    int c = 0x494a4b5c;
    f(&b);
    printf("%08x %08x %08x\n", a, b, c);
    return 0;
}

I compiled this on linux with 'gcc -g' and ran under gdb and got this:

Program received signal SIGSEGV, Segmentation fault.
0x000000000040056a in f (p=0x7fffffffde74) at smash.c:10
10  }
(gdb) bt
#0  0x000000000040056a in f (p=0x7fffffffde74) at smash.c:10
#1  0x3636363600400594 in ?? ()
#2  0x3030303035353535 in ?? ()
#3  0x494a4b5c31313131 in ?? ()
#4  0x0000000000000000 in ?? ()
(gdb)

As you can see, the parent function addresses now contain some of my magic numbers. I ran this on 64 bit linux, so really I should have used 64 bit ints to fully overwrite the return address - as it is I left the lower word untouched.

Answer 2

Try the function below, I wrote it for 32-bit compiler try using ( -m32 gcc flag) or with a little effort you can make it work with your 64-bit compiler (Note that in your objdump listing you got 7 bytes offset between call to function and the next instruction so use 7 instead of 8 .

void function(void)
{
    unsigned long *x;
    /* &x will more likely be at -4(ebp) */
    /* Adding 1 (+4) gets us to stored ebp */
    /* Adding 2 (+8) gets us to stored return address */
    x = (unsigned long *)(&x + 2);

    /* This is the tricky part */
    /* TODO: On my 32-bit compiler gap between call to function
       and the next instruction is 8 */
    *x += 8;
}