如何处理来自 MIPI 相机的 12 位图像？

Question

Actually, I use a Gige camera. That camera provide images in unsigned char* , this mean a bit depth of 8. I develop in c/c++.

I am going to receive a new one, but that camera is different. This one provides void* , with a bit depth of 12. Can I cast the void* like that:

unsigned short* img ;
img = (unsigned short*) camera.GetImage() ; /**camera.GetImage() is void*/

It is new for me, and I have been never using raw format. Does cast is a good idea? I need to access at each pixel, and I don't how handle pixel with void* .

I suggest to cast with unsigned short* because of the depth. The pixel of the camera are encoded on 12. If I cast with unsigned char* , I would lose information.

Thank you for your help !

UPDATE

Thank you a lot for all your details. As I am a beginner, your answer help me a lot to find other ways to investigate.

I am working on jetson TX2, and the bit location is a reflection of the actual hardware.

For my part, the pixel data is concatenated in 16bits. The output of the jetson TX2 is:

              ONE PIXEL

|              16 bits                          |
D5-D4-D3-D2-D1-D0-D11-D10-0-0-D11-D10-D9-D8-D7-D6

My goal is not to display the image. Below the explanation of my process:

1 - Take 5 times the same image;

2 - Calculation of the average image;

3 - Bad pixels correction;

4 - Flat field correction;

4 - Send final image through socket;

In a thread, I acquire images. In an other thread, I apply image processing and in an other one, I send image through socket. I implemented all the process.

I didn't use libraries like openCV because I did some specifics process. I need to handle all pixels of the image, and I wonder if it is very efficient to convert the .raw in unsigned short* , to adapt the image to my code.

The 'packing' was never a question because my previous camera provides images in an unsigned char* . Do you think that I should convert the void* into unsigned short* ?

Thank you for your help

Answer 1

There is not enough information in your post to answer your question. Here are the possible packing for 12-bits pixels:

Efficient packing

Bit0     Bit11     Bit23      Bit35        Bit47     Bit59     Bit71     Bit83      Bit95
[ Pixel0   | Pixel1  | Pixel2    | Pixel3  | Pixel4   | Pixel5  | Pixel6  |  Pixel7 ]  (that is 8 pixels per 96bits or 3 32-bits word) 

Easy to use packing

Bit0    Bit11     Bit23    Bit32     Bit43     Bit55     Bit63     
[ Pixel0   | Pixel1  | 0    | Pixel3  | Pixel4   | 0      ]  (that is 2 pixels per 32-bits word) 

Dumb packing

Bit0    Bit11     Bit16    Bit27     Bit31          
[ Pixel0   |  0    | Pixel1  |  0      ]  (that is 1 pixels per 16-bits word) 

You have to read the datasheet of your camera to figure which packing is being used. Once this is done, you also need to figure out the endianness of the data for transmitting the data words. For example, in a little endian system, you'll the least significant byte first. For a big endian system, you'll get the most significant byte first. You'll need to adapt your pixel extraction code to fit your machine's endianness.

Extracting pixel is done quite simply once you have figured out the packing and endianness with bit shifting and mask.

Since you have asked for a C++ solution, here's a any-bit extractor from a memory stream, you'll need to adjust for the packing and endianness:

struct BitStream
{
    const uint8 * begin;
    const uint8 * end;
    size_t pos;
    BitStream(const uint8 * bits, size_t len) : begin(bits), end(bits + len), pos(0) {}
    // get value from bitstream coded on len bits
    uint32 get(uint8 len)
    {
        size_t byte_offset = pos/8;
        uint8 bit_pos = pos % 8;
        uint8 bit_avail = 8 - bit_pos;
        if (begin+byte_offset >= end) return 0;
        if (bit_avail >= len)
        {
            uint32 value = begin[byte_offset];
            uint8 mask = ~(0xff << len) << bit_pos;
            value = value & mask;
            value = value >> bit_pos;
            pos = pos + len;
            return value;
        }
        else
        {
            uint32 value = get(bit_avail);
            value = value | (get(len - bit_avail)<<bit_avail);
            return value;
        }
    }
    void skip(uint8 len) { pos += len; }
    bool done() const { return (begin + pos/8) >= end; }
};

If there is endianness mismatch between the camera and your machine, you'll need to swap the returned pixel like this:

BitStream bs((const uint8*)camera.getImage(), camera.getImageLen());
// Get pixel
uint16 pixel = (uint16)bs.get(12);
// Comment the line below if the endianness is the same
pixel = ((pixel & 0xFF00) >> 8) | (pixel << 8);
// Skip the stuffing zero bits depending on packing format, here I assume dumb packing
bs.skip(4); 

Answer 2

Thank you for updating your question.

If, as it now seems, each 12-bit pixel is fully contained in a 16-bit unsigned short with 4-bits of padding, then "yes" unsigned short is definitely the way to go. I don't know if your environment has uint16_t or not but that is guaranteed to be exactly 16-bits (which I feel is better) whereas unsigned short is "at least" 16-bits.

Now you just need to do some ANDing and shifting...