如何在Python中将2通道的光流灰度JPG图像合并为一个RGB图像？

Question

I'm making a RGB image from the given 2 grayscale jpg images, which is for x, y channel in optical flow respectively.

sample input images and my current output

def optical_flow(one, two, w, h):
    """
    method taken from (https://chatbotslife.com/autonomous-vehicle-speed-estimation-from-dashboard-cam-ca96c24120e4)
    """
    one_g = cv2.cvtColor(one, cv2.COLOR_RGB2GRAY)
    two_g = cv2.cvtColor(two, cv2.COLOR_RGB2GRAY)
    hsv = np.zeros((w, h, 3))
    # set saturation
    hsv[:,:,1] = cv2.cvtColor(two, cv2.COLOR_RGB2HSV)[:,:,1]
    # obtain dense optical flow paramters
    flow = cv2.calcOpticalFlowFarneback(one_g, two_g, flow=None,
                                        pyr_scale=0.5, levels=1, winsize=15,
                                        iterations=2,
                                        poly_n=5, poly_sigma=1.1, flags=0)
    # convert from cartesian to polar
    mag, ang = cv2.cartToPolar(flow[..., 0], flow[..., 1])
    # hue corresponds to direction
    hsv[:,:,0] = ang * (180/ np.pi / 2)
    # value corresponds to magnitude
    hsv[:,:,2] = cv2.normalize(mag,None,0,255,cv2.NORM_MINMAX)
    # convert HSV to int32's
    hsv = np.asarray(hsv, dtype= np.float32)
    rgb_flow = cv2.cvtColor(hsv,cv2.COLOR_HSV2RGB)
    return rgb_flow 

imgu = cv2.imread('u.jpg')
imgv = cv2.imread('v.jpg')
img = optical_flow(one, two, w, h)

Image.fromarray(imgu, 'RGB').show() // top-left one in img
Image.fromarray(imgv, 'RGB').show() // top-right one in img
Image.fromarray(img, 'RGB').show()  // bottom-left one in img

I think the output image looks strange.

Answer 1

Optical flow computes a motion vector field of two consequtive frames. In you case one and two . However, your input images u and v does not show consequtive frames. The problem here is that there is no relation between the images, ie no similar content. Cosequently your optical flow field flow will have some random like values.

When computing the color coded visualisation of your optical flow field. In your case the saturation channel is initialized not correctly:

# set saturation
hsv[:,:,1] = cv2.cvtColor(two, cv2.COLOR_RGB2HSV)[:,:,1]

Set it to 255. More common solutation is to set the value channel to 255 and use the saturation channel to encode the magnitude.