How does the assembly TEST instruction work with these jump instructions?

Question

Using AT&T assembly syntax, I'm trying to understand how testl is used in assembly code. Specifically:

testl  %edx, %edx
jle    .L3

I know testl does a bitwise and of the same value to set the condition flags, but how can I interpret 'jump if less than or equal to' if it isn't comparing two values?

Answer 1

Here's an excerpt from the official documentation from Intel on test:

Operation
TEMP ← SRC1 AND SRC2;
SF ← MSB(TEMP);
IF TEMP = 0
    THEN ZF ← 1;
    ELSE ZF ← 0;
FI:
PF ← BitwiseXNOR(TEMP[0:7]);
CF ← 0;
OF ← 0;

And the same on jle:

Jump if less or equal (ZF=1 or SF≠OF)

So, the jump will be performed if edx is 0 (because edx AND edx = edx and that's 0 only when edx is 0, and because ZF is set to 1 when the result of AND is 0) or if the most significant bit of edx is 1 (because SF = most significant bit of edx AND edx (or, equivalently, of edx itself) and OF is always 0, which means SF ≠ OF is only true when SF ≠ 0 ).

IOW, the jump will be performed only if edx is ≤ 0 when interpreted as a signed integer or, equivalently, when edx is either 0 or greater or equal than 0x80000000 when interpreted as an unsigned integer.

Answer 2

In x86 assembly almost all conditional jumps are based on flags (except jcxz , jecxz , loop and loopne / loopnz ). This means that all that matters are the values the flags have.

jle is synonymous with jng . The jump condition is ZF = 1 or SF <> OF . You may want to check Intel x86 JUMP quick reference .

test does set all flags except AF link , so everything looks good so far.

According to this link logical operations always zero OF . That means your jump would practically be ZF = 1 or SF = 1 , so in your code jle would jump if edx was 0 or between range 0x80000000 ... 0xffffffff .

Answer 3

TESTL with identical arguments (like edx and edx ) sets the flags based on the value of that argument itself (since x AND x is identical to x). So we can forget about the AND altogether here since it's discarded - all we need to concern ourselves with is the value in edx .

With TESTL , the zero flag ZF is set to 1 only if the value is zero. TESTL also forces the overflow flag OF to 0 and sets the sign flag SF only if the high bit is set.

JLE will then jump if either ZF is set to 1, or SF <> OF .

So, the jump will execute if either:

• edx was zero; or
• edx had its high bit set.

Hence it will jump for edx values of 0 or 0x80000000 - 0xffffffff .

Most likely this is a check to ensure that the number is a natural number 0x00000001 - 0x7fffffff , the jump would be to an error handling routine of some sort and a valid natural number would continue without the jump, something like:

loop_for_number:
    call   get_number_into_edx
    testl  %edx, %edx
    jle    loop_for_number

    ; carry on here knowing that edx >= 1

---

For a description of the various jumps and the flags they use, see here .