NASM - JBE instruction gives segmentation fault

Question

I am trying to implement a fibonacci algorithm in assembly using the NASM assembler. This is the pseudo code for the algorithm

fibonacci_it(n):
    int f_i-1 = 1;
    int f_i-2 = 1;

    if (n==0):
        return 1;
    else:
        int i = 1;
        int f_i = 1;
        while (i < n):
            f_i = f_i-1 + f_i-2
            f_i-2 = f_i-1
            f_i-1 = f_i
            i = i + 1
        return f_i

What I've tried looks like this:

%include "asm_io.inc"

segment .data
prompt      db  "Enter number: ", 0

segment .bss
fibnum      resd 1

segment .text
    global asm_main

asm_main:
    enter   0,0
    pusha   

    mov         eax, prompt
    call        print_string
    call        read_int
    mov         [fibnum], eax

    push        dword [fibnum]
    call        fib_it
    call        print_int

    popa
    mov         eax, 0
    leave
    ret



fib_it:
    enter   12,0                ; 3 local variables: f_i, f_i-1, f_i-2
    pusha

    mov     dword [ebp-4], 1    ; initialize f_i
    mov     dword [ebp-8], 1    ; initialize f_i-1
    mov     dword [ebp-12], 1   ; initialize f_i-2
    mov     ecx, 1              ; i = 1
    mov     edx, 1              ; comparison operator for n
    cmp     [ebp+8], edx        ; n <= 1 ?
    jbe     end_fib_it          ; if n <= 1 -> end and return 1

fib_it_while:
    dump_regs 1
    mov     eax, [ebp-8]        ; eax = f_i-1
    add     eax, [ebp-12]       ; eax = f_i-1 + f_i-2
    mov     [ebp-4], eax        ; f_i = f_i-1 + f_i-2
    mov     eax, [ebp-8]        ; eax = f_i-1
    mov     [ebp-12], eax       ; f_i-2 = f_i-1
    mov     eax, [ebp-4]        ; eax = f_i
    mov     [ebp-8], eax        ; f_i-1 = f_i
    inc     ecx                 ; i += 1
    cmp     ecx, [ebp+8]        ; i < n ?
    jae     end_fib_it          ; end while loop if i < n
    jmp     fib_it_while        ; else -> loop again

end_fib_it:
    mov     eax, [ebp-4]        ; return f_i
    popa
    leave
    ret

The program first reads an integer from the terminal. Then it pushes this to the stack as parameter for fib_it When I run the program it gives a segmentation fault. I found out that every time jbe end_fib_it or jae end_fib_it is true and the program has to jump then it gives a segmentation fault. I've made some more debugging with dump_regs and found out that the while loop runs perfectly and jmp fib_it_while gives no problems.

Answer 1

I found out that every time jbe end_fib_it or jae end_fib_it is true and the program has to jump then it gives a segmentation fault.

Your observation might not be the problem at all.

 push dword [fibnum] call fib_it 

When you call fib_it, you push a dword on the stack and you don't pop it anywhere afterwards.

The simplest of solutions is probably to end fib_it with an alternate form of ret :

popa
leave
ret 4      ; The 4 will remove the parameter from the stack

---

Don't use pusha / popa

 end_fib_it: mov eax, [ebp-4] ; return f_i popa 

You want to return a result in EAX , but the popa instruction that follows will immediately destroy what you've just put in there!

Since you're only using the additional registers ECX and EDX , best change pusha into push ecx push edx . Also change popa into pop edx pop ecx .

---

Optimization

 mov edx, 1 ; comparison operator for n cmp [ebp+8], edx ; n <= 1 ? 

Why would you use the additional register EDX if all you're using it for is a single comparison?
If you change this pair of instructions into:

cmp     dword [ebp+8], 1        ; n <= 1 ?

the forementioned replacement for pusha / popa will only be push ecx / pop ecx .