如何在c或c ++中改进多维位数组比较性能

Question

I have the following three-dimensional bit array(for a bloom filter):

unsigned char  P_bit_table_[P_ROWS][ROWS][COLUMNS];

the P_ROWS's dimension represents independent two-dimensional bit arrays(ie, P_ROWS[0], P_ROWS 1 ,P_ROWS[2] are independent bit arrays) and could be as large as 100MBs and contains data which are populated independently. The data that I am looking for could be in any of these P_ROWS and right now I am searching through it independently, which is P_ROWS[0] then P_ROWS 1 and so on until i get a positive or until the end of it(P_ROWS[n-1]). This implies that if n is 100 I have to do this search(bit comparison) 100 times(and this search is done very often). Some body suggested that I can improve the search performance if I could do bit grouping (use a column-major order on the row-major order array-- I DON'T KNOW HOW).

I really need to improve the performance of the search because the program does a lot of it.

I will be happy to give more details of my bit table implementation if required.

Sorry for the poor language.

Thanks for your help.

EDIT: The bit grouping could be done in the following format: Assume the array to be :

unsigned char P_bit_table_[P_ROWS][ROWS][COLUMNS]={{(a1,a2,a3),(b1,b2,b3),(c1,c2,c3))},
                                                  {(a1,a2,a3),(b1,b2,b3),(c1,c2,c3))},   
                                                  {(a1,a2,a3),(b1,b2,b3),(c1,c2,c3))}};

As you can see all the rows --on the third dimension-- have similar data. What I want after the grouping is like; all the a1's are in one group(as just one entity so that i can compare them with another bit for checking if they are on or off ) and all the b1's are in another group and so on.

Answer 1

Re-use Other People's Algorithms

There are a ton of bit-calculation optimizations out there including many that are non-obvious, like Hamming Weights and specialized algorithms for finding the next true or false bit, that are rather independent of how you structure your data.

Reusing algorithms that other people have written can really speed up computation and lookups, not to mention development time. Some algorithms are so specialized and use computational magic that will have you scratching your head: in that case, you can take the author's word for it ( after you confirm their correctness with unit tests ).

Take Advantage of CPU Caching and Multithreading

I personally reduce my multidimensional bit arrays to one dimension, optimized for expected traversal.

This way, there is a greater chance of hitting the CPU cache.

In your case, I would also think deeply about the mutability of the data and whether you want to put locks on blocks of bits. With 100MBs of data, you have the potential of running your algorithms in parallel using many threads, if you can structure your data and algorithms to avoid contention.

You may even have a lockless model if you divide up ownership of the blocks of data by thread so no two threads can read or write to the same block. It all depends on your requirements.

Now is a good time to think about these issues. But since no one knows your data and usage better than you do, you must consider design options in the context of your data and usage patterns.