使用Interop处理字节序的标准是什么？

Question

I created a very simple DLL for Speck in (granted, probably inefficient) ASM. I connected to it in C# using InteropServices.

When I tested this crypto with the test vectors provided in the paper describing the algorithm , I found that the only way to get them to come out right was to "flip" the key and the plain text, and then to "flip" the crypto at the end for a match. So an endianness issue I guess. I have seen the same, for example, between a reference implementation of Serpent and TrueCrypt's version -- they produce the same result only with the bytes in the reverse order.

I will post my assembly code and my C# code for reference, though it may not be critical to see the code in order to understand my question. In the C# code is a click event handler that checks the DLL for consistency with the test vectors. As you can also see there, the program has to do a lot of array flipping in that handler to get the match.

So the question I have been working towards is this. Should I "flip" those arrays inside the DLL to account for endianness? Or should I leave it to the caller (also me, but C# side)? Or am I making mountains out of molehills and I should just ignore endianness at this point? I am not planning to sell the silly thing, so there is no worry about compatibility issues, but I am a stickler for doing things right, so I am hoping you all can guide me on the best practice here if there is one.

ASM:

.code ; the beginning of the code
 ; section
WinMainCRTStartup proc h:DWORD, r:DWORD, u:DWORD ; the dll entry point
 mov rax, 1 ; if eax is 0, the dll won't
 ; start
 ret ; return
WinMainCRTStartup Endp ; end of the dll entry

_DllMainCRTStartup proc h:DWORD, r:DWORD, u:DWORD ; the dll entry point
 mov rax, 1 ; if eax is 0, the dll won't
 ; start
 ret ; return
_DllMainCRTStartup Endp                                 

SpeckEncrypt proc plaintText:QWORD, cipherText:QWORD, Key:QWORD
; Pass in 3 addresses pointing to the base of the plainText, cipherText, and         Key arrays
; These come in as RCX, RDX, and R8, respectively
; I will use These, RAX, and R9 through R15 for my working space.  Will do 128 bit block, 128 bit key sizes, but they will fit nicely in 64 bit registers

; simple prologue, pushing ebp and ebx and the R# registers, and moving the value of esp into ebp for the duration of the proc  
push rbp
mov rbp,rsp
push rbx
push R9
push R10
push R11
push R12
push R13
push R14
push R15

; Move data into the registers for processing
mov r9,[rcx] ; rcx holds the memory location of the first 64 bits of plainText.  Move this into R9.  This is plainText[0] 
mov r10,[rcx+8] ; put next 64 bits into R10.  This is plainText[1]
;NOTE that the address of the cipherText is in RDX but we will fill r11 and r12 with values pointed at by RCX.  This is per the algorithm.  We will use RDX to output the final bytes
mov r11,[rcx] ; cipherText[0] = plainText[0]
mov r12,[rcx+8] ; cipherText[1] = plainText[1] 
mov r13, [r8] ;First 64 bits of key.  This is Key[0]
mov r14, [r8+8] ; Next 64 bits of key.  This is Key[1]

push rcx ; I could get away without this and loop in another register, but I want to count my loop in rcx so I free it up for that
mov rcx, 0 ; going to count up from here to 32.  Would count down but the algorithm uses the counter value in one permutation, so going to count up

EncryptRoundFunction:
ror r12,8
add r12,r11
xor r12,r13
rol r11,3
xor r11,r12

ror r14,8
add r14,r13
xor r14,rcx
rol r13,3
xor r13,r14

inc rcx
cmp rcx, 32
jne EncryptRoundFunction

pop rcx
; Move cipherText into memory pointed at by RDX.  We won't bother copying the Key or plainText back out
mov [rdx],r11
mov [rdx+8],r12

; Now the epilogue, returning values from the stack into non-volatile registers.
pop R15
pop R14
pop R13
pop R12
pop R11
pop R10
pop R9    
pop rbx    
pop rbp
ret ; return eax
SpeckEncrypt endp ; end of the function

SpeckDecrypt proc cipherText:QWORD, plainText:QWORD, Key:QWORD
; Pass in 3 addresses pointing to the base of the cipherText, plainText, and Key arrays
; These come in as RCX, RDX, and R8, respectively
; I will use These, RAX, and R9 through R15 for my working space.  Will do 128 bit block, 128 bit key sizes, but they will fit nicely in 64 bit registers

; simple prologue, pushing ebp and ebx and the R# registers, and moving the value of esp into ebp for the duration of the proc  
push rbp
mov rbp,rsp
push rbx
push R9
push R10
push R11
push R12
push R13
push R14
push R15

; Move data into the registers for processing
mov r9,[rcx] ; rcx holds the memory location of the first 64 bits of cipherText.  Move this into R9.  This is cipherText[0] 
mov r10,[rcx+8] ; put next 64 bits into R10.  This is cipherText[1]
;NOTE that the address of the plainText is in RDX but we will fill r11 and r12 with values pointed at by RCX.  This is per the algorithm.  We will use RDX to output the final bytes
mov r11,[rcx] ; plainText[0] = cipherText[0]
mov r12,[rcx+8] ; plainText[1] = cipherText[1] 
mov r13, [r8] ;First 64 bits of key.  This is Key[0]
mov r14, [r8+8] ; Next 64 bits of key.  This is Key[1]

push rcx ; I could get away without this and loop in another register, but I want to count my loop in rcx so I free it up for that
mov rcx, 0 ; We will count up while making the round keys

DecryptMakeRoundKeys:
; On encrypt we could make each key just as we needed it.  But here we need the keys in reverse order.  To undo round 31 of encryption, for example, we need round key 31.

; So we will make them all and push them on the stack, pop them off again as we need them in the main DecryptRoundFunction
; I should pull this off and call it for encrypt and decrypt to save space, but for now will have it separate

; push r13 at the beginning of the process because we need a "raw" key by the time we reach decrypt round 0
; We will not push r14 because that half of the key is only used here in the round key generation function.
; We don't need it in the decrypt rounds
push r13

ror r14,8
add r14,r13
xor r14,rcx
rol r13,3
xor r13,r14

inc rcx
cmp rcx, 32
jne DecryptMakeRoundKeys

mov rcx, 32
DecryptRoundFunction:
dec rcx
pop r13

xor r11,r12
ror r11,3
xor r12,r13
sub r12,r11
rol r12,8

cmp rcx, 0
jne DecryptRoundFunction


pop rcx
; Move cipherText into memory pointed at by RDX.  We won't bother copying the Key or plainText back out
mov [rdx],r11
mov [rdx+8],r12

; Now the epilogue, returning values from the stack into non-volatile registers.
pop R15
pop R14
pop R13
pop R12
pop R11
pop R10
pop R9    
pop rbx    
pop rbp
ret ; return eax
SpeckDecrypt endp ; end of the function

End ; end of the dll

And the C#:

using System;
using System.Runtime.InteropServices;
using System.Text;
using System.Threading;
using System.Windows.Forms;

namespace SpeckDLLTest
{
    public partial class Form1 : Form
    {
        byte[] key = { 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 };
        public Form1()
        {
            InitializeComponent();
            Array.Reverse(key);
        }

        private void richTextBox1_TextChanged(object sender, EventArgs e)
        {
            textBox1.Text = richTextBox1.Text.Length.ToString();
            if (richTextBox1.Text != "")
            {
                byte[] plainText = ASCIIEncoding.ASCII.GetBytes(richTextBox1.Text);
                byte[] cipherText = new byte[plainText.Length];

                Thread t = new Thread(() =>
                {
                    cipherText = Encrypt(plainText);
                    BeginInvoke(new Action(() => richTextBox2.Text = Convert.ToBase64String(cipherText)));
                });
                t.Start();
                t.Join();
                t.Abort();


                byte[] plainAgain = new byte[cipherText.Length];
                t = new Thread(() =>
                    {
                        plainAgain = Decrypt(cipherText);
                        BeginInvoke(new Action(() => richTextBox3.Text = ASCIIEncoding.ASCII.GetString(plainAgain)));
                    });
                t.Start();
                t.Join();
                t.Abort();
            }
            else
            {
                richTextBox2.Text = "";
                richTextBox3.Text = "";
            }
        }

        private byte[] Decrypt(byte[] cipherText)
        {
            int blockCount = cipherText.Length / 16;
            if (cipherText.Length % 16 != 0) blockCount++;
            Array.Resize(ref cipherText, blockCount * 16);
            byte[] plainText = new byte[cipherText.Length];
            unsafe
            {
                fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
                {
                    for (int i = 0; i < blockCount; i++)
                    {
                        for (int j = 0; j < 1; j++)
                        {
                            UnsafeMethods.SpeckDecrypt(ciphertextPointer + i * 16, plaintextPointer + i * 16, keyPointer);
                        }
                    }
                }
            }
            return plainText;
        }

        private byte[] Encrypt(byte[] plainText)
        {
            int blockCount = plainText.Length / 16;
            if (plainText.Length % 16 != 0) blockCount++;
            Array.Resize(ref plainText, blockCount * 16);
            byte[] cipherText = new byte[plainText.Length];
            unsafe
            {
                fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
                {
                    for (int i = 0; i < blockCount; i++)
                    {
                        for (int j = 0; j < 1; j++)
                        {
                            UnsafeMethods.SpeckEncrypt(plaintextPointer + i * 16, ciphertextPointer + i * 16, keyPointer);
                        }
                    }
                }
            }
            return cipherText;
        }

        private void button1_Click(object sender, EventArgs e)
        {
            byte[] plainText = { 0x6c, 0x61, 0x76, 0x69, 0x75, 0x71, 0x65, 0x20, 0x74, 0x69, 0x20, 0x65, 0x64, 0x61, 0x6d, 0x20 };
            byte[] key = { 0x0f, 0x0e, 0x0d, 0x0c, 0x0b, 0x0a, 0x09, 0x08, 0x07, 0x06, 0x05, 0x04, 0x03, 0x02, 0x01, 0x00 };
            byte[] testVector = { 0xa6, 0x5d, 0x98, 0x51, 0x79, 0x78, 0x32, 0x65, 0x78, 0x60, 0xfe, 0xdf, 0x5c, 0x57, 0x0d, 0x18 };

            Array.Reverse(key);
            Array.Reverse(plainText);

            byte[] cipherText = new byte[16];
            unsafe
            {
                fixed (byte* plaintextPointer = plainText, ciphertextPointer = cipherText, keyPointer = key)
                {
                    UnsafeMethods.SpeckEncrypt(plaintextPointer, ciphertextPointer, keyPointer);
                    Array.Reverse(cipherText);
                    bool testBool = true;
                    for (int i = 0; i < cipherText.Length; i++)
                    {
                        if (testVector[i] != cipherText[i]) testBool = false;
                    }
                    if (testBool == false) MessageBox.Show("Failed!");
                    else MessageBox.Show("Passed!");
                }
            }
        }
    }

    public static class UnsafeMethods
    {
        [DllImport("Speck.dll")]
        unsafe public extern static void SpeckEncrypt(byte* plainText, byte* cipherText, byte* Key);
        [DllImport("Speck.dll")]
        unsafe public extern static void SpeckDecrypt(byte* cipherText, byte* plainText, byte* Key);
    }
}

Answer 1

Whether someone might like it or not, the de facto standard for byte order when it comes to networking and cryptography is big-endian (most significant byte first — the “natural” order). This applies not only to serialization of data for inter-system exchange, but to intra-system API as well and for any other case where caller is not supposed to be aware of callee internals. This convention does not have anything to do with endianness of particular hardware and popularity of such hardware. It just sets the default format for exchanged data, so that both lower-level and higher-level programs may pass data around without regard to their degree of awareness of what this data contains and how it is processed.

However, if the caller is supposed to be tightly coupled with the callee, it may be more convenient and performance-wise to pass the data in a more preprocessed form, especially if some of that data remains constant across invocations. For example, if we are dealing with asymmetric cryptography, it may be easier and faster to call the core functions with all data already translated to big integers, and for those we may prefer little-endian digit order (a “digit” or a “limb” is usually a half of largest available register) even on a big-endian byte order hardware — simply because such an order of digits is more useful for arbitrary-precision math library. But those details should not be visible to the outside world — for anyone else, we are accepting and returning big-endian bytestream.

---

Regarding your specific task.

1. As @RossRidge already pointed out, you are probably very wrong if your are simply flipping entire arrays, — you should swap bytes ( BSWAP ) in particular pieces being processed rather than inverting the order of those pieces besides that.
2. Chances are high that you are very overestimating your ability to write efficient machine code: for example, you don't interleave instructions with unrelated registers for better out-of-order execution, your loop is not aligned, you use counter increase to N instead of decrease to zero. Of course, that code will still be 10x faster than .Net anyway, but I strongly recommend you to write an implementation in C and benchmark — to get amazed of how good a compiler (MSVC, GCC) may be at optimizing even a straight-though written program (believe me, I once committed the same mistake when trying to accomplish the same task). If performance is not a big issue, do not mess with unmanaged code at all, — because it is just an external non-portable dependency that increases required trust level for you .Net application.
3. Use .Net functions dealing with bytes with caution, because they are very inconsistent with regard to endianness: BitConverter uses host byte order, StreamReader always sticks to little-endian, and String is all about the encoding given (of all UTF encodings, only UTF-8 is endian-agnostic).

That are the issues I noticed at first glance. There may be more of them.