How can I send captured video stream frames from html to flask server?

Question

I have 3 main code(camera.py, main.py, xx.html) run perfectly in my local server. But I when uploaded them to heroku, application is not working since cv2.VideoCapture isn't triggering webcam on client side.

Main.py:

from flask import Flask, render_template,Response,request,jsonify
from camera import VideoCamera
from flask import redirect,url_for
import os
from PIL import Image
from mains.utils import pipeline_model
from wtforms import Form, BooleanField, StringField, PasswordField, validators
from predictors import RegressionPredictor, CNNPredictor
import numpy as np


UPLOAD_FOLDER ='static/upload'

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

def gen(camera):
    while True:
        frame = camera.get_frame()
        yield (b'--frame\r\n'

               b'Content-Type: image/jpeg\r\n\r\n' + frame + b'\r\n\r\n')

@app.route('/video_feed')
def video_feed():
    return Response(gen(VideoCamera()),
                    mimetype='multipart/x-mixed-replace; boundary=frame')

if __name__ == '__main__':

    app.run(debug=True)

camera.py:

import cv2


face_cascade=cv2.CascadeClassifier("haarcascade_frontalface_alt2.xml")
ds_factor=0.6


class VideoCamera(object):
    def __init__(self):
        self.video = cv2.VideoCapture(0)

    def __del__(self):
        self.video.release()


    def get_frame(self):

        ret,frame = self.video.read()
        frame = cv2.resize(frame,None,fx=ds_factor,fy=ds_factor,interpolation=cv2.INTER_AREA)
        gray = cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
        face_rects = face_cascade.detectMultiScale(gray,1.3,5)
        for (x,y,w,h) in face_rects:
            cv2.rectangle(frame,(x,y),(x+w,y+h),(0,255,0),2)
            break

        ret,jpeg = cv2.imencode('.jpg',frame)
        return jpeg.tobytes()

.html code (only related part)

<img id="bg" src="{{url_for('video_feed')}}" alt="">

As I mentioned above code opens my webcam on my browser and detects my face.

I searched and found some javascript code to reach web cam on the client side. Code is like below;

<!doctype html>

<html lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <title>WebcamJS Test Page</title>

<body>


    <h1>WebcamJS Test Page</h1>
    <h3>Demonstrates simple 320x240 capture &amp; display</h3>

    <div id="my_camera"></div>

    <!-- First, include the Webcam.js JavaScript Library -->
    <script type="text/javascript" src="static/webcam.min.js"></script>

    <!-- Configure a few settings and attach camera -->
    <script language="JavaScript">
        Webcam.set({
            width: 320,
            height: 240,
            image_format: 'jpeg',
            jpeg_quality: 90
        });
        Webcam.attach( '#my_camera' );
    </script>

</body>
</html>

This code works perfectly but I can not figured out how can I send data frames from "Webcam.attach('#my_camera') to flask server to process it using OpenCV? I searched a lot but couldn't find solution maybe its because I am newbie on javascript.

Thanks in advance for your support!

Answer 1

SocketIO was the answer to make connection between server and javascript code. Please check this repo for more information.

Please check this one to get OpenCV integrated version of above code

How can I send captured video stream frames from html to flask server?

Question

1 answers

solution1
1 2020-12-20 15:33:31

How can I send captured video stream frames from html to flask server?

Question

1 answers

solution1 1 2020-12-20 15:33:31

solution1
1 2020-12-20 15:33:31