Fast byte array masking in C#

Question

I have a struct with some properties (like int A1 , int A2 ,...). I store a list of struct as binary in a file.

Now, I'm reading the bytes from file using binary reader into Buffer and I want to apply a filter based on the struct's properties (like .A1 = 100 & .A2 = 12 ).

The performance is very important in my scenario, so I convert the filter criteria to byte array ( Filter ) and then I want to mask Buffer with Filter . If the result of masking is equal to Filter , the Buffer will be converted to the struct.

The question: What is the fastest way to mask and compare two byte arrays?

Update: The Buffer size is more than 256 bytes. I'm wondering if there is a better way rather than iterating in each byte of Buffer and Filter .

Answer 1

Try a simple loop with System.BitConverter.ToInt64() . Something Like this:

byte[] arr1;
byte[] arr2;

for (i = 0; i < arr1.Length; i += 8) 
{
    var P1 = System.BitConverter.ToInt64(arr1, i);
    var P2 = System.BitConverter.ToInt64(arr2, i);

    if((P1 & P2) != P1) //or whatever
        //break the loop if you need to.
}

My assumption is that comparing/masking two Int64 s will be much faster (especially on 64-bit machines) than masking one byte at a time.

Answer 2

The way I would usually approach this is with unsafe code. You can use the fixed keyword to get a byte[] as a long* , which you can then iterate in 1/8th of the iterations - but using the same bit operations. You will typically have a few bytes left over (from it not being an exact multiple of 8 bytes) - just clean those up manually afterwards.

Answer 3

Once you've got the two arrays - one from reading the file and one from the filter, all you then need is a fast comparison for the arrays. Check out the following postings which are using unsafe or PInvoke methods.

What is the fastest way to compare two byte arrays?

Comparing two byte arrays in .NET