Optimizing Bayer16 -> RGB in Python

Question

I am reading a camera that gives me Bayer16 format (GRGB) and I wrote the following code in python to modify it from bayer16 to bayer8, and then use OpenCV to convert it to RGB:

def _convert_GRGB_to_RGB(self, bayer16_image):
  bayer8_image = bytearray()
  # Convert bayer16 to bayer 8
  for i in range(0, len(bayer16_image), 2):
    data_byte = (bayer16_image[i] & 0xF0) >> 4
    data_byte |= (bayer16_image[i+1] & 0x0F) << 4
    bayer8_image.append(data_byte)
  bayer8_image = numpy.frombuffer(bayer8_image, dtype=numpy.uint8).reshape((720, 1280))
  # Use OpenCV to convert Bayer GRGB to RGB
  return cv2.cvtColor(bayer8_image, cv2.COLOR_BayerGR2RGB)

After doing some timing, the for loop takes most of the running time and is extremely inefficient (although I think it does not allocate any space, unless numpy makes a copy for very edit). I am wondering how to improve this function as a whole, or the for loop in particular (as it is the slowest part of this function, by an order of magnitude).

Does anyone have tips and advice about how to improve this Bayer16 -> RGB conversion if I am to use Python please?

EDIT:

I found a solution using numpy array that makes my code pretty fast:

def _convert_GRGB_to_RGB(self, data_bytes):
  even = numpy.frombuffer(data_bytes[0::2], dtype=numpy.uint8)
  odd = numpy.frombuffer(data_bytes[1::2], dtype=numpy.uint8)
  # Convert bayer16 to bayer8
  even = numpy.right_shift(even, 4)
  odd = numpy.left_shift(odd, 4)
  bayer8_image = numpy.bitwise_or(even, odd).reshape((720, 1280))
  # Use OpenCV to convert Bayer GRGB to RGB
  return cv2.cvtColor(bayer8_image, cv2.COLOR_BayerGR2RGB)

This solution satisfies my need but if anyone has any suggestion, I'm curious to hear them!

Answer 1

Your can use standard python operators in your numpyified code, You'll also get a speedup by not slicing data_bytes (assuming it's bytes and not itself a numpy array)

def _convert_GRGB_to_RGB(self, data_bytes):
    data_bytes = numpy.frombuffer(data_bytes, dtype=numpy.uint8)
    even = data_bytes[0::2]
    odd = data_bytes[1::2]
    # Convert bayer16 to bayer8
    bayer8_image = (even >> 4) | (odd << 4)
    bayer8_image = bayer8_image.reshape((720, 1280))
    # Use OpenCV to convert Bayer GRGB to RGB
    return cv2.cvtColor(bayer8_image, cv2.COLOR_BayerGR2RGB)

Answer 2

As a guess, your color problem is as follows - your GRBG data comes in like this:

G0 B1 G2 ...
R0 G1 R2

Where the numbers represent the uint16 index. OpenCV needs them to be numbered

G0 B0 G1 R1 ...
R6 G6 R7 G7

You can fix this with some careful reshape and transposing:

data_bytes = np.frombuffer(data_bytes, dtype=np.uint8)
data = data.reshape(height / 2, width, 2) # a pair for each uint16
data = data.transpose((0, 2, 1))  #move the G/RB axis to be adjacent to the height axis
data = data.reshape(height, width)  # collapse it

Example

# manually constructed by hand
sample = ''.join([
    'grbGgRbGgRbg'
    'grBGGRBGGRbg'
    'grBgGrBgGrbg'
])
width = height = 6
data = np.array(list(sample))

data = (data
    .reshape(height / 2, width, 2)
    .transpose((0, 2, 1))
    .reshape(height, width)
)

# easy way to view the output
>>> data.view((np.str_,6))
array([['gbgbgb'],
       ['rGRGRg'],
       ['gBGBGb'],
       ['rGRGRg'],
       ['gBGBGb'],
       ['rgrgrg']], 
      dtype='<U6')