什么是使用GCC重排大量位向量的最有效方法

Question

I have two very big bit vectors (about 1 GB each) and I want to shuffle them in the following fashion:

first bit vector: a[0], a[1], a[n]
second bit vector: b[0], b[1], b[n]

It should result in something like:

c[0] = a[0]
c[1] = b[0] 
c[2] = a[1]
c[3] = b[1]

What's the most efficient way to do that in C++, using the vector operations of the new Intel processors? I want to do this using GCC.

Answer 1

you could try rolling your own loop --

int ch1, ch2;
while ((ch1 = fgetc(fp1)) != EOF && (ch2 = fgetc(fp2)) != EOF) {
    int i, dst = 0;
    // assuming msb goes first
    for (i=7; i>=0; i--) {
        dst |= (ch1 & (1<<i)) << (2*i + 1);
        dst |= (ch2 & (1<<i)) << (2*i + 0);
    }
    putc(dst >> 8);
    putc(dst & 0xFF);
}

You can massage this a bit, unroll it, prefetch chunks into local arrays, process 16 bits in the loop, but it interleaves the bits in two bytes in 4 instructions per source bit (-O3 unrolled the loop).

If we assume two bytes take 150 cycles on a 3GHz processor, that's 40 MB/sec output from 2x20 MB/sec source data read, or 50 seconds for 2x1000 MB. Feeding data to the loop, however, may cut into the throughput.