OpenCV frame by frame video stitching

Question

I am trying to stitch the input from two distinct cameras set with different angles. What should I do in order to merge the output of the cameras (each frame one by one) into a single frame?

When I get the input as the previously recorded videos ("InputVideo1.mp4" and "InputVideo2.mp4" as commented below), frames generated from the video inputs stitch. However, when I enable the cameras as inputs, OpenCV Stitcher returns enum "ERR_NEED_MORE_IMGS = 1".

// Create a VideoCapture object and open the input file
//VideoCapture vCap("InputVideo1.mp4");
//VideoCapture vCap1("InputVideo2.mp4");

// If the input is the web camera, pass 0 instead of the video file name
VideoCapture vCap(0);
VideoCapture vCap1(1);

// Check if the camera opened successfully
if (!vCap.isOpened()) {
    cout << "Error opening video stream or file" << endl;
    return -1;
}
if (!vCap1.isOpened()) {
    cout << "Error opening video stream or file" << endl;
    return -2;
}

while (1) {

    Mat frame, frame1, finalFrame;

    // Capture frame-by-frame
    vCap >> frame;
    vCap1 >> frame1;

    // If the frame is empty, break immediately
    if (frame.empty()) {
        cout << "Frame is empty" << endl;
        break;
    }
    if (frame1.empty()) {
        cout << "Frame1 is empty" << endl;
        break;
    }

    // Display the resulting frame
    imshow("First frame", frame);
    imshow("Second frame", frame1);
    
    // Push each frame one by one into the final vector 
    vector<Mat> finalFrameImages;

    finalFrameImages.push_back(frame);
    finalFrameImages.push_back(frame1);


    // Stitch all frames that are stored the final frame images vector set
    Stitcher::Mode mode = Stitcher::PANORAMA;
    Ptr<Stitcher> stitcher = Stitcher::create(mode);
    Stitcher::Status status = stitcher->stitch(finalFrameImages, finalFrame);

    if (status != Stitcher::OK) {
        cout << "Can't stitch images" << endl;
        return status;
    }
    

    imshow("Resulting frame", finalFrame);

}

Stitcher::Status status should return 0 but it returns 1.

Answer 1

It might be that the pictures are the same. Take a look at this diagram: Now we see that in the registration block the method look at features between two images and matches them in order to do the stitching correctly. Would suggest you to check if you import good images before doing the stitching, I can't see other problem. Other than that check out this link