如何在Python中將實時數據讀入循環與更多處理密集型操作分開？

Question

因此，我有一個OpenCV網絡攝像頭供稿，希望盡快讀取它的幀。 由於使用了Python GIL，腳本在幀中讀取的最快速度似乎如下：

#Parent or maybe client(?) script

#initilize the video capture object
cam  = cv2.VideoCapture(0)

while True:
    ret, frame = cam.read()

    # Some code to pass this numpy frame array to another python script 
    # (which has a queue) that is not under time constraint and also
    # is doing some more computationally intensive post-processing...

    if exit_condition is True:
        break

我想發生的是將這些幀（Numpy數組）添加到第二個Python腳本（或可能是多處理實例）中的某種處理隊列中，然后執行一些不在像cam.read（）循環這樣的時間限制是...

因此，基本思路如下所示：

實時（或盡快獲得）數據收集（相機讀取）腳本---->分析腳本（將進行后處理，寫入結果並生成滯后於數據收集的matplotlib圖）

我已經做過一些研究，看起來像：管道，套接字，pyzmq和python多處理都可以實現我想要的東西。 問題是我沒有上述任何經驗。

因此，我的問題是，哪種方法最能實現我所尋找的目標，並且有人可以提供一個簡短的示例，甚至是一些想法/想法來向我指出正確的方向嗎？

非常感謝！

編輯：非常感謝史蒂夫讓我開始正確的道路。 這是我所想到的一個工作要點...代碼本身可以正常工作，但是如果添加更多的后處理步驟，則隊列大小可能會比主進程可以通過的速度更快。 限制幀速率的建議很可能會成為我最終使用的策略。

import time
import cv2
import multiprocessing as mp

def control_expt(connection_obj, q_obj, expt_dur):

    def elapsed_time(start_time):
        return time.clock()-start_time

    #Wait for the signal from the parent process to begin grabbing frames
    while True:
        msg = connection_obj.recv()     
        if msg == 'Start!':
            break    

    #initilize the video capture object
    cam  = cv2.VideoCapture(cv2.CAP_DSHOW + 0)  

    #start the clock!!
    expt_start_time = time.clock() 

    while True:
        ret, frame = cam.read()          
        q_obj.put_nowait((elapsed_time(expt_start_time), frame))

        if elapsed_time(expt_start_time) >= expt_dur:
            q_obj.put_nowait((elapsed_time(expt_start_time),'stop'))
            connection_obj.close()
            q_obj.close()
            cam.release()
            break

class test_class(object):
    def __init__(self, expt_dur):

        self.parent_conn, self.child_conn = mp.Pipe()
        self.q  = mp.Queue()
        self.control_expt_process = mp.Process(target=control_expt, args=(self.child_conn, self.q, expt_dur))
        self.control_expt_process.start()

    def frame_processor(self):
        self.parent_conn.send('Start!')

        prev_time_stamp = 0

        while True:
           time_stamp, frame = self.q.get()                        
           #print (time_stamp, stim_bool)          
           fps = 1/(time_stamp-prev_time_stamp)
           prev_time_stamp = time_stamp      

            #Do post processing of frame here but need to be careful that q.qsize doesn't end up growing too quickly...
            print (int(self.q.qsize()), fps)

            if frame == 'stop':
                print 'destroy all frames!'
                cv2.destroyAllWindows()
                break               
            else:
                cv2.imshow('test', frame)        
                cv2.waitKey(30)

        self.control_expt_process.terminate()

if __name__ == '__main__':  
    x = test_class(expt_dur = 60)
    x.frame_processor()

Answer 1

多處理文檔是一個很好的起點。 https://docs.python.org/2/library/multiprocessing.html#exchanging-objects-between-processes我建議您閱讀此書，即使您現在可能還不了解。

在您提到的其他技術上使用管道可以使您保持性能並保持代碼簡單。

以下是一些我尚未測試過的代碼，它們應該為您提供一個良好的起點。

from multiprocessing import Process, Pipe

def read_frames(connection_obj):
  #initilize the video capture object
  cam  = cv2.VideoCapture(0)
  while True:
    ret, frame = cam.read()
    connection_obj.send(frame) # is this what you want to send?

    if exit_condition is True:
        connection_obj.close()
        break

def launch_read_frames(connection_obj):
    """
    Starts the read_frames function in a separate process.
    param connection_obj: pipe to send frames into.
    Returns a pipe object
    """
    read_frames_process = Process(target=read_frames, args=(connection_obj,)) # this trailing comma is intentional
    read_frames_process.start()
    read_frames_process.join()

    return parent_conn

def act_on_frames(connection_obj):
    while True:
        frame = connection_obj.recv()
        # Do some stuff with each frame here

if __name__ == '__main__':
    parent_conn, child_conn = Pipe()
    launch_read_frames(child_conn)

    # You could also call this function as a separate process, though in
    # this instance I see no performance benefit.
    act_on_frames(parent_conn)