Checking if a char is equal to multiple other chars, with as little branching as possible

Question

I'm writing some performance-sensitive C# code that deals with character comparisons. I recently discovered a trick where you can tell if a char is equal to one or more others without branching, if the difference between them is a power of 2.

For example, say you want to check if a char is U+0020 (space) or U+00A0 (non-breaking space). Since the difference between the two is 0x80, you can do this:

public static bool Is20OrA0(char c) => (c | 0x80) == 0xA0;

as opposed to this naive implementation, which would add an additional branch if the character was not a space:

public static bool Is20OrA0(char c) => c == 0x20 || c == 0xA0;

How the first one works is since the difference between the two chars is a power of 2, it has exactly one bit set. So that means when you OR it with the character and it leads to a certain result, there are exactly 2 ^ 1 different characters that could have lead to that result.

Anyway, my question is, can this trick somehow be extended to characters with differences that aren't multiples of 2? For example, if I had the characters # and 0 (which have a difference of 13, by the way), is there any sort of bit-twiddling hack I could use to check if a char was equal to either of them, without branching?

Thanks for your help.

edit: For reference, here is where I first stumbled across this trick in the .NET Framework source code, in char.IsLetter . They take advantage of the fact that a - A == 97 - 65 == 32 , and simply OR it with 0x20 to uppercase the char (as opposed to calling ToUpper ).

Answer 1

If you can tolerate a multiply instead of a branch, and the values you are testing against only occupy the lower bits of the data type you are using (and therefore won't overflow when multiplied by a smallish constant, consider casting to a larger data type and using a correspondingly larger mask value if this is an issue), then you could multiply the value by a constant to force the two values to be a power of 2 apart.

For example, in the case of # and 0 (decimal values 35 and 48), the values are 13 apart. Rounding down, the nearest power of 2 to 13 is 8, which is 0.615384615 of 13. Multiplying this by 256 and rounding up, to give an 8.8 fixed point value gives 158.

Here are the binary values for 35 and 48, multiplied by 158, and their neighbours:

34 * 158 = 5372 = 0001 0100 1111 1100
35 * 158 = 5530 = 0001 0101 1001 1010
36 * 158 = 5688 = 0001 0110 0011 1000

47 * 158 = 7426 = 0001 1101 0000 0010
48 * 158 = 7548 = 0001 1101 1010 0000
49 * 158 = 7742 = 0001 1110 0011 1110

The lower 7 bits can be ignored because they aren't necessary in order to separate any of the neighbouring values from each other, and apart from that, the values 5530 and 7548 only differ in bit 11, so you can use the mask and compare technique, but using an AND instead of an OR. The mask value in binary is 1111 0111 1000 0000 (63360) and the compare value is 0001 0101 1000 0000 (5504), so you can use this code:

public static bool Is23Or30(char c) => ((c * 158) & 63360) == 5504;

I haven't profiled this, so I can't promise it's faster than a simple compare.

If you do implement something like this, be sure to write some test code that loops through every possible value that can be passed to the function, to verify that it works as expected.

Answer 2

You can use the same trick to compare against a set of 2^N values provided that they have all other bits equal except N bits. Eg if the set of values is 0x01, 0x03, 0x81, 0x83 then N=2 and you can use (c | 0x82) == 0x83 . Note that the values in the set differ only in bits 1 and/or 7. All other bits are equal. There are not many cases where this kind of optimization can be applied, but when it can and every little bit of extra speed counts, its a good optimization.

This is the same way boolean expressions are optimized (eg when compiling VHDL). You may also want to look up Karnaugh maps.

That being said, it is really bad practice to do this kind of comparisons on character values especially with Unicode, unless you know what you are doing and are doing really low level stuff (such as drivers, kernel code etc). Comparing characters (as opposed to bytes) has to take into account the linguistic features (such as uppercase/lowercase, ligatures, accents, composited characters etc)

On the other hand if all you need is binary comparison (or classification) you can use lookup tables. With single byte character sets these can be reasonably small and really fast.

Answer 3

If not having branches is really your main concern, you can do something like this:

if ( (x-c0|c0-x) & (x-c1|c1-x) & ... & (x-cn|cn-x) & 0x80) {
  // x is not equal to any ci

If x is not equal to a specific c, either xc or cx will be negative, so xc|cx will have bit 7 set. This should work for signed and unsigned chars alike. If you & it for all c's, the result will have bit 7 set only if it's set for every c (ie x is not equal to any of them)