C/C++ Inline asm improper operand type

Question

I have the following code, that is supposed to XOR a block of memory:

void XorBlock(DWORD dwStartAddress, DWORD dwSize, DWORD dwsKey)
{
DWORD dwKey;
__asm
{
    push eax
    push ecx
    mov ecx, dwStartAddress          // Move Start Address to ECX
    add ecx, dwSize                  // Add the size of the function to ECX
    mov eax, dwStartAddress          // Copy the Start Address to EAX

    crypt_loop:                         // Start of the loop
        xor byte ptr ds:[eax], dwKey     // XOR The current byte with 0x4D
        inc eax                         // Increment EAX with dwStartAddress++
        cmp eax,ecx                     // Check if every byte is XORed
    jl crypt_loop;                      // Else jump back to the start label

    pop ecx // pop ECX from stack
    pop eax // pop EAX from stack
}
}

However, the argument dwKey gives me an error. The code works perfectly if for example the dwKey is replaced by 0x5D.

Answer 1

I think you have two problems.

First, "xor" can't take two memory operands (ds:[eax] is a memory location and dwKey is a memory location); secondly, you've used "byte ptr" to indicate you want a byte, but you're trying to use a DWORD and assembly can't automatically convert those.

So, you'll probably have to load your value into an 8-bit register and then do it. For example:

void XorBlock(DWORD dwStartAddress, DWORD dwSize, DWORD dwsKey)
{
    DWORD dwKey;
    __asm
    {
        push eax
        push ecx
        mov ecx, dwStartAddress          // Move Start Address to ECX
        add ecx, dwSize                  // Add the size of the function to ECX
        mov eax, dwStartAddress          // Copy the Start Address to EAX
        mov ebx, dwKey                   // <---- LOAD dwKey into EBX

        crypt_loop :                         // Start of the loop
            xor byte ptr ds : [eax], bl     // XOR The current byte with the low byte of EBX
            inc eax                         // Increment EAX with dwStartAddress++
            cmp eax, ecx                     // Check if every byte is XORed
            jl crypt_loop;                      // Else jump back to the start label

        pop ecx // pop ECX from stack
        pop eax // pop EAX from stack
    }
}

Although, it also looks like dwKey is uninitialized in your code; maybe you should just "mov bl, 0x42". I'm also not sure you need to push and pop the registers; I can't remember what registers you are allowed to clobber with MSVC++ inline assembler.

But, in the end, I think Alan Stokes is correct in his comment: it is very unlikely assembly is actually faster than C/C++ code in this case. The compiler can easily generate this code on its own, and you might find the compiler actually does unexpected optimizations to make it run even faster than the "obvious" assembly does (for example, loop unrolling ).