Combining cv2.imshow() with matplotlib plt.show() in real time

Question

I'm trying to combine a feed from webcam using openCV, and then updating a graph using matplotlib.

For getting and showing the frames a basic example:

import cv2

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, frame = cap.read()

    # Display the resulting frame
    cv2.imshow('frame',frame)

    # When to exit loop - terminate program
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()

An example of continuously updating a graph (plotting randomly) with matplotlib:

import numpy as np
import matplotlib.pyplot as plt

x = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
y = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

# x goes from 0-9 numbers
# y goes from 0-100%
fig = plt.figure()
ax = plt.axes(xlim=(0, 9), ylim=(0, 100))
# line, = ax.plot([], [], lw=2)
rects = plt.bar(x, y, color='b')

def animate(i):
    y = random.sample(xrange(100), 10)
    for rect, yi in zip(rects, y):
        rect.set_height(yi)
    return rects

anim = animation.FuncAnimation(fig, animate,
                           frames=200, interval=20, blit=True)

plt.show()

So what I want is to combine the two together. The graph should be updated by passing results that I obtain from the frames. The major problem I am facing is getting both windows to update simultaneously side by side. The plt.show() seems to be blocking everything else.

Any idea on how to resolve?

Cheers

Answer 1

Here's an example of converting a plt.figure() to np.array and show it along camera feed with cv2.imshow

import matplotlib
matplotlib.use('TkAgg')

import numpy as np
import cv2
import matplotlib.pyplot as plt

fig = plt.figure()
cap = cv2.VideoCapture(0)


x1 = np.linspace(0.0, 5.0)
x2 = np.linspace(0.0, 2.0)

y1 = np.cos(2 * np.pi * x1) * np.exp(-x1)
y2 = np.cos(2 * np.pi * x2)


line1, = plt.plot(x1, y1, 'ko-')        # so that we can update data later

for i in range(1000):
    # update data
    line1.set_ydata(np.cos(2 * np.pi * (x1+i*3.14/2) ) * np.exp(-x1) )

    # redraw the canvas
    fig.canvas.draw()

    # convert canvas to image
    img = np.fromstring(fig.canvas.tostring_rgb(), dtype=np.uint8,
            sep='')
    img  = img.reshape(fig.canvas.get_width_height()[::-1] + (3,))

    # img is rgb, convert to opencv's default bgr
    img = cv2.cvtColor(img,cv2.COLOR_RGB2BGR)


    # display image with opencv or any operation you like
    cv2.imshow("plot",img)

    # display camera feed
    ret,frame = cap.read()
    cv2.imshow("cam",frame)

    k = cv2.waitKey(33) & 0xFF
    if k == 27:
        break