从 OpenCV 捕获设备捕获 YUV 帧

Question

I need to extract YUV frames directly from a web camera using OpenCV from C++ on the Windows platform. In other words: a setup in OpenCV that makes the capture device's read() method return a YUV Mat. I'm looking for a working example or documentation on how to do this.

The specific YUV subformat isn't that important for starters. The camera can produce YUV output. I cannot simply convert a BGR Mat to YUV - I need to capture it from the device without any conversions.

So far I've tried and combined

• connecting using several specific APIs (CV_CAP_DSHOW, CV_CAP_FFMPEG, CV_CAP_GSTREAMER, etc.)
• changing between different FOURCCs (MJPG, UYVY, etc.)
• toggling the CAP_PROP_CONVERT_RGB setting
• to find out what CAP_PROP_MODE might do

All to no avail as I'm still just getting output in BGR. So, any hints, links, three-liners?

(I'm using OpenCV 4.5.2)

Answer 1

You suppose to use the CAP_PROP_CONVERT_RGB.

From the docs :

Boolean flags indicating whether images should be converted to RGB. GStreamer note: The flag is ignored in case if custom pipeline is used. It's user responsibility to interpret pipeline output.

There a lot of issues with this flag. In this post the author added a fix for FFMPEG:

void cv_VideoCapture_setConvertToRGB(
        cv::VideoCapture& cap,
        const bool convertFormat)
{
    cap.set(cv::CAP_PROP_CONVERT_RGB, convertFormat);
#if CV_VERSION_MAJOR >= 4
    const cv::VideoCaptureAPIs backend =
            static_cast<cv::VideoCaptureAPIs>(qRound(cap.get(cv::CAP_PROP_BACKEND)));
    switch (backend) {
    case cv::CAP_MSMF:
        // Note in case something needs to change here:
        // MSMF backend uses fourcc codes for format and uses
        //  CAP_PROP_FORMAT and CAP_PROP_FOURCC interchangeably
        break;
    case cv::CAP_FFMPEG:
        // See cap_ffmpeg_impl.hpp
        cap.set(cv::CAP_PROP_FORMAT, convertFormat ? 0 : -1);
        break;
    default:
        break;
    }
#endif // CV_VERSION_MAJOR >= 4
}

Answer 2

Seems that OpenCV under windows is made with Direct Show which cannot grab YUV variants directly. So the solution is not to use OpenCV but Windows Media Foundation. OpenCV under other OS'es may be able to grab YUV.