OpenCV how to encode and send data of webcam video stream efficiently?

Question

I have this OpenCV C++ code which is taking an image from the cam, encoding it and sending it to the STDOUT.

#include <unistd.h> //STDOUT_FILENO
#include "opencv2/opencv.hpp"
#include <iostream>
#include <fcntl.h>
using namespace std;
using namespace cv;

#define BUFLEN 4096

int main(int argc, char *argv[])
{
    Mat frame;
    std::vector<uchar> buf;

    int bak, temp;

    //read image as grayscale
    namedWindow( "Camera", WINDOW_AUTOSIZE );
    //redirect stdout to NULL in order to avoid printing to STDOUT undesired stuff
    fflush(stdout);
    bak = dup(1);
    temp = open("/dev/null", O_WRONLY);
    dup2(temp, 1);
    close(temp  );
    VideoCapture cam(0 + CAP_V4L);
    cam>>frame;
    sleep(1);
    if (!cam.isOpened())
    {
        cout << "\nCould not open reference " << 0 << endl;
        return -1;
    }
    for (int i=0; i<5; i++)
    {
        cam>>frame;
    }
    /*Set the normal STDOUT back*/
    fflush(stdout);
    dup2(bak, 1);
    close(bak);

    //encode image and put data into the vector buf
    imencode(".png",frame, buf);
    //send the total size of vector to parent
    cout<<buf.size()<<endl;
    unsigned int written= 0;

    int i = 0;
    size_t toWrite = 0;
    //send until all bytes have been sent
    FILE * f = fdopen(STDOUT_FILENO, "w");
    while (written<buf.size())
    {
        //send the current block of data
        toWrite = BUFLEN < (buf.size()-written) ? BUFLEN : (buf.size()-written);
        //written += write(STDOUT_FILENO, buf.data()+written, toWrite);
        written += toWrite*fwrite ( buf.data()+written, toWrite, 1, f );
        i++;
    }
    return 0;
}

Now instead of an image I would like to take an infinite continuous video from the cam. One solution would be to take a frame any given seconds, encode the frame and transmit it (print it to STDOUT), all actions inside an infinite loop.

Is there a better solution, more efficient than encoding and send each frame at each iteration?

Answer 1

Fundamentally, a video stream is a sequence of frames in a predefined order.

You could simply send the frames as images, one after another. There is nothing fundamentally wrong with that, but is not necessarily optimal (which also depends on your definition of optimal).

In communication, one of the aspects is to minimize the amount of data transferred. Simply sending the frames as images allows you to do some compression (eg jpeg). Better compression algorithms for video (eg mpeg) use the temporal properties of the sequence. If there is a (mostly) static frame, you can limit yourself sending data about the changing parts and assume the background is the same. This goes with some processing at both ends of the communication, but might increase the communication speed (assuming the link is the bottleneck). This would also add a lot of complexity to the system, so think about the potential advantages first (identify the bottlenecks).

I am not sure about the usage of your application, but sending the video stream to stdout might not necessarily be the best idea. Consider using a pipe or a socket instead (the latter allows you quite easily to transfer data over the network as well, which might be a nice outcome).