Stack Overflow when computing n = ~11000 + fibonacci recursive method with memoization

Question

I have a program to compute the nth Fibonacci number with a recursive method and while using memoization.

I get up to around n = 11000 and I am getting a stackoverflow exception. Can someone help me fix this?

Here is my code:

public class BigInt {

static BigInteger[] fval;

public static void main(String[] args) {
    int index;
    Scanner input = new Scanner(System.in);
    index = input.nextInt();
    fval = new BigInteger[index + 1];

    System.out.println(fib_rec(index));
}

public static BigInteger fib_rec(int index) {
    BigInteger result = BigInteger.ONE;
    if (index <= 2) {
        return result;
    } else {
        if (fval[index] != null) {
            result = fval[index];
        } else {
            result = fib_rec(index - 1).add(fib_rec(index - 2));
            fval[index] = result;
        }
        return result;
    }
}

Answer 1

Your memoization doesn't change the depth of recursion... on the call to fib_rec(n) , it calls fib_rec(n-1) which calls fib_rec(n-2) etc. If you reverse the order of calls (so that you do fib_rec(index - 2).add(fib_rec(index - 1)) that should allow you to cut the stack depth roughly in half since you'd be working your way down to the root by twos and then fill in the gaps from the bottom up with a stack depth of one thanks to your memoization.

There's no way, however, to avoid the stack depth issue without a more dramatic rewrite of the algorithm.

Answer 2

The Inevitable

The StackOverflowError is inherently unfixable for large enough input values. The argument for this is two-folded. First, Java does not have tail call optimization . Second, with each recursive call, Java has to allocate some memory, eg for parameters. Even if your method has no parameters, Java needs a little bit of memory to store the address to jump to after the method call has ended. Thus, you eventually will exhaust your memory, no matter how large. You can prolong the inevitable by starting the JVM with more stack memory. The option (as well as some others) can be found here .

Can we do better?

Yes, we can. But not with a recursive algorithm. We need to turn this recursive algorithm in an iterative algorithm. In fact, each recursion can be transformed in an iteration ans vice-versa . This alone is still not sufficient since your algorithm has linear memory complexity. We actually need only two values to calculate the next Fibonacci number. This leads to the following approach (pseudo-code):

int fibonacci(int nth)
    if nth is smaller than 0
        then halt and catch fire

    if nth is smaller than 2
        then return nth

    int last <- 1
    int secondToLast <- 0;

    for (int ith <- 2; ith less or equal to nth, increment ith)
        int tmp <- last + secondToLast
        secondToLast <- last
        last <- tmp
    end for

    return last;

The above algorithm has linear time complexity (given that addition can be done in constant time) and constant memory complexity, thus solves your problem.

Answer 3

Avoid recursion while using memoization. Here is an example:

public class BigInt {

    public static void main(String[] args) {
        Scanner input = new Scanner(System.in);
        int index = input.nextInt();

        System.out.println(fib_rec(index));
    }

    public static BigInteger fib_rec(int index) {
        if (index <= 2) {
            return BigInteger.ONE;
        }

        BigInteger[] fval = new BigInteger[index + 1];
        fval[0] = BigInteger.ONE;
        fval[1] = BigInteger.ONE;

        for (int i = 2; i <= n; i++) {
            fval[i] = fval[i-1].add(fval[i-2]);
        } 

        return fval[n];
   }
}

}

Answer 4

The reason you are getting a stackoverflow is because you are running out of memory. Increase the memory available and more specifically the stack size. Just add -Xss1024mb or whatever size you prefer.

The best way to handle this kind of situation is to actually have a better algorithm so that you dont need to consume alot of memory.