OpenCV 无法从视频中读取图像帧

Question

I am currently trying to use OpenCV with Python to load a video from a url onto a localhost webpage. The loaded video is a little choppy but the main problem is that it stops reading the video frames after a while and displays the following error message.

[h264 @ 0955e140] error while decoding MB 87 29, bytestream -5
[h264 @ 0955e500] left block unavailable for requested intra4x4 mode -1
[h264 @ 0955e500] error while decoding MB 0 44, bytestream 126
Debugging middleware caught exception in streamed response at a point where response headers were already sent.
Traceback (most recent call last):
  File "C:\Users\\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\werkzeug\wsgi.py", line 506, in __next__
    return self._next()
  File "C:\Users\\AppData\Local\Programs\Python\Python38-32\Lib\site-packages\werkzeug\wrappers\base_response.py", line 45, in _iter_encoded
    for item in iterable:
  File "C:\Users\\Downloads\VideoStreamingFlask\main.py", line 12, in gen
    frame = camera.get_frame()
  File "C:\Users\\Downloads\VideoStreamingFlask\camera.py", line 13, in get_frame
    ret, jpeg = cv2.imencode('.jpg', image)
cv2.error: OpenCV(4.3.0) C:\projects\opencv-python\opencv\modules\imgcodecs\src\loadsave.cpp:919: error: (-215:Assertion failed) !image.empty() in function 'cv::imencode'

Code

main.py

from flask import Flask, render_template, Response
from camera import VideoCamera

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

def gen(camera):
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'
               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_feed')
def video_feed():
    return Response(gen(VideoCamera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':
    app.run(host='0.0.0.0', debug=True)

camera.py

import cv2

class VideoCamera(object):
    def __init__(self):
        self.video = cv2.VideoCapture(*url*)

    def __del__(self):
        self.video.release()
    
    def get_frame(self):
        success, image = self.video.read()
        ret, jpeg = cv2.imencode('.jpg', image)
        return jpeg.tobytes()

Questions

1. What might be causing the problem here?
2. How do I make the video less choppy?

Answer 1

The crash in your python happened because video.read() failed. Therefore, image can not be passed to cv2.imencode() . You should check the success value in get_frame(self) and be prepared that sometimes camera.get_frame() will not return a valid Jpeg.

Now, let's understand why video.read() failed in this case. This could happen if the connection to the camera was not good enough and some packets got lost. But more likely, your VideoCapture was not fast enough to handle the video stream.

This could be improved if you reduce the work that the video capture thread is doing. As suggested in another discussion , offload processing to a separate thread.

Currently, your flask server listens to camera stream, converts it to a series of Jpegs, and sends them to the client over HTTP. If you have a thread dedicated to camera stream, you may find that your server cannot pass every video frame, because the encoder and HTTP transport are too slow. So, some frames will be skipped.

Here is a detailed article about video streaming with flask: https://blog.miguelgrinberg.com/post/flask-video-streaming-revisited . You can find some other open-source projects that stream video to browser, not necessarily with opencv and python.