inline assembly segmentation fault

Question

all,

I was trying to write a rot13 with inline assembly function...

the following code works with 'a', but when c goes to 'z', it doesn't work anymore...

and... it always shows "segmentation fault" ... Please give me some advice to solve

this problem.

#include <stdio.h>

#define add(a,b)\
asm volatile(\
    "add %%ebx,%%eax" \
    :"=a"(a) \
    :"a"(a),"b"(b) \
)

#define rot13(a)\
asm (\
"rot:\n\t"\
    "add $13,%%eax\n\t"\
    "cmpl $64,%%eax\n\t"\
    "jle L5f\n\t"\
    "cmpl $90, %%eax\n\t"\
    "jg L5f\n\t"\
    "cmpl $90,%%eax\n\t"\
    "jle L5f\n\t"\
    "subl $26,%%eax\n\t"\
"L5f:\n\t"\
    "cmpl $96,%%eax\n\t"\
    "jle L6f\n\t"\
    "cmpl $122,%%eax\n\t"\
    "jg L6f\n\t"\
    "cmpl $122,%%eax\n\t"\
    "jle L6f\n\t"\
    "subl $26,%%eax\n\t" \
"L6f:\n\t"\
    "leave\n\t"\
    :"=r"(a)\
    :"r"(a)\
)

int main()
{
    int a=13, b=12,c='z';
    rot13(c);
    printf("c-> rot13= %c\n",c);

    return 0;
}

Answer 1

I bet your segmentation fault is because you have a leave instruction in there. Inline assembly does not get called like a normal function, so you don't need that, and it'll mangle the stack.

Another problem is that you have all your assembly hardcoded to operate on %eax , but you didn't tell the compiler that. It thinks it can stick the input in any register it wants and read the output back from any other register it wants. It's possible to adapt the code to that by using %0 and %1 instead of %%eax all the time, but it's probably easier to change the input and output constraints to

asm ( <your code here> : "=a" (a) : "0" (a) )

which means "the output of this inline assembly must be in %eax , and the input must be in the same register as the output". (If you put "a" again on the input constraint it will not work correctly. GCC's register allocator is a 28-year-old pile of hacks upon hacks and you have to play by its rules. The rules may be found in the " Extended Asm " and " Constraints for Asm Operands " sections of the GCC manual ; read them very carefully , including all subsections of the second section, and keep in mind that this is actually a repurposed feature of the internal "machine description" language and is optimized for that.)

This still doesn't give me the right answer for rot13('z'), but I think the remaining problems are bugs in your rot13 algorithm rather than in its interface with the rest of the program.