C overflow function.. segmentation fault

Question

When I try to access secret function in the below code, compiler gives segm fault. I achieved access, but after reach the secret function retr_addr was doNotTouch[12]'s address. I can not reach return 0;

How can I solve this problem, thanks.

Output:
now inside entrance()!
now inside secret()!

Process finished with exit code 11

#include <stdio.h>

void secret()
{
    printf("now inside secret()!\n");
}

void entrance()
{
    int doNotTouch[4];
    // can only modify this section BEGIN
    // cant call secret(), maybe use secret (pointer to function)


    // printf("addr_doNot = %d\n", &(doNotTouch));
    // *(doNotTouch + 10) = (int) &secret;

    void * a = secret;
    doNotTouch[10] = a;

    // can only modify this section END
    printf("now inside entrance()!\n");
}

int main (int argc, char *argv[])
{
    entrance();
    return 0;
}

Answer 1

This is possible, but depends on the compilation options. I assume that you're using x86-64.

1. In unoptimized builds, you have control of SP via BP. When entrance() leaves, using the leave instruction, it sets SP to BP, and pops BP, and finally returns and goes to secret() 's address that you've conveniently put on the stack. Then secret() can return to main - again, you'd put the address in the correct location on the stack. At this point, the SP is 8 bytes too high, since we've returned through one function "too many". But when main is done, it will set SP to BP, and thus restore the proper stack pointer and BP, and then return, popping the return address from the correct stack location. The sketch of the code is, then:

    void **stk = (void**)&stk; stk +=...; // adjust stk[1] = stk[0]; // forward return point into main to secret stk[0] = &secret; // return into secret

2. In optimized builds, stack frames are suppressed, and you don't have this ability to control SP nor to automatically restore it. But since SP is not modified on function return, then returning from functions moves SP up, and calling into functions moves it down. You'd need to grow the stack a bit by recursing into entrance , and detecting that by scanning a bit of stack above to see if a magic value is stored from the outer invocation. After detecting recursion, you'd load the stack with return address to secret, above that with repeated addresses of some benign library function like abs that uses a register calling convention and can be used to move SP up. Eventually, this function would return to main normally, with SP at the correct position. But this assumes that you're allowed to call entrance again.

3. Alternatively, if you can't explicitly call entrance, you have to shift enough of the stack up by one void* size, so that when secret returns to main, main and outer functions will find the return addresses where they expected them. This is the extension of the stk[n+1] = stk[n] idea - you'll need to strategically choose the addresses to shift so that main has a valid return address, as well as whatever runtime functions without stack frames that call main (since ones with stack frames will "fix" themselves as long as you shift the first BP they need up on the stack).

4. Yet alternatively, you could look up various lexicons of return-oriented programming, and write a small return-oriented program that would adjust the stack pointer as desired.

For x86-64, using gcc 10 (and many other versions), I've got the approach 1+3 working. This also works to an extent on clang, ie it crashes in optimized builds when returning from main ; there's a slightly different layout of stack in main in clang, so that would require some tweaking, potentially shifting two discontinuous chunks of stack.

The dump_stack function is there only for debugging this hack, it's not called and can be removed if you wish. Feel free to play with it on godbolt .

#include <stdio.h>
#include <stdlib.h>

void dump_stack(void **p, int len)
{
    p += len-1;
    while (len--) {
        printf("%16llx: %16llx\n", (long long)p, (long long)*p);
        --p;
    }
}

void secret()
{
    printf("now inside secret()!\n");
}

void entrance()
{
    int doNotTouch[4];
    // can only modify this section BEGIN

    #define stk (*(void*volatile*volatile*)&doNotTouch)
    stk = (void*)&doNotTouch;

    #ifndef __OPTIMIZE__
        // gcc w/o optimizations
        enum { first = 6, last = 5 };
    #else
        // gcc -O & up
    #if 1
        // when dump_stack is not called
        enum { first = 8, last = 3 };
    #else
        // only needed for -O1,2,3 when dump_stack is called
        enum { first = 8, last = 5};
    #endif
    #endif
    for (int i = first; i > last; --i)
        stk[i] = stk[i-1];
    stk[last] = &secret;

    // dump_stack(stk, 20);

    // can only modify this section END
    printf("now inside entrance()!\n");
}

int main(int argc, char *argv[])
{
    entrance();
    return 0;
}

Answer 2

if you want to invoke function using pointer you need pointer to function, not an array

the problem with your code is you are trying to store address in a normal array and invoke, that does not work, the array accessing is also not correct, you are accessing beyond allocated size. *(doNotTouch + 10) = (int) &secret;

#include <stdio.h>

void secret()
{
    printf("now inside secret()!\n");
}


void entrance()
{
    int doNotTouch[4];
    void (*f)(void);
    // can only modify this section BEGIN
    // cant call secret(), maybe use secret (pointer to function)


    // printf("addr_doNot = %d\n", &(doNotTouch));
    // *(doNotTouch + 10) = (int) &secret;

    //void * a = secret;
    //doNotTouch[10] = a;
    
    f = secret;
    f();
    // can only modify this section END
    printf("now inside entrance()!\n");
}

int main (int argc, char *argv[])
{
    entrance();
    return 0;
}