sws_scale, YUV to RGB conversion

Question

I need convert YUV to RGB. I also need the RGB values to be in the limited range (16-235). I try to use sws_scale function for this task.

My code you can see below. But after conversion I got the black pixel is (0, 0, 0) instead of (16, 16, 16).

Maybe there are some options to tell sws_scale function to calculate the limited range.

AVFrame* frameRGB = avFrameConvertPixelFormat(_decodedBuffer[i].pAVFrame, AV_PIX_FMT_RGB24);

AVFrame* Decoder::avFrameConvertPixelFormat(const AVFrame* src, AVPixelFormat dstFormat) {
    int width = src->width;
    int height = src->height;

    AVFrame* dst = allocPicture(dstFormat, width, height);

    SwsContext* conversion = sws_getContext(width,
                                            height,
                                            (AVPixelFormat)src->format,
                                            width,
                                            height,
                                            dstFormat,
                                            SWS_FAST_BILINEAR,
                                            NULL,
                                            NULL,
                                            NULL);
    sws_scale(conversion, src->data, src->linesize, 0, height, dst->data, dst->linesize);
    sws_freeContext(conversion);

    dst->format = dstFormat;
    dst->width = src->width;
    dst->height = src->height;

    return dst;
}

Also I tried convert YUV pixel to RGB pixel manualy with formula and I got correct result. From YUV (16, 128, 128) I got (16, 16, 16) in RGB.

cmpR = y + 1.402 * (v - 128);
cmpG = y - 0.3441 * (u - 128) - 0.7141 * (v - 128);
cmpB = y + 1.772 * (u - 128);

Answer 1

You may the source format to "full scale" YUVJ .

As far as I know, sws_scale has no option for selecting Studio RGB as output format.
Changing the input format is the best solution I can think of.

The color conversion formula of "JPEG: YUV -> RGB" is the same as the formula in your post.

Examples for setting the source format:

• If src->format is PIX_FMT_YUV420P , set the format to PIX_FMT_YUVJ420P .
• If src->format is PIX_FMT_YUV422P , set the format to PIX_FMT_YUVJ422P .
• If src->format is PIX_FMT_YUV444P , set the format to PIX_FMT_YUVJ444P .
• If PIX_FMT_YUV440P , use PIX_FMT_YUVJ440P .

I know the solution is not covering all the possibilists, and there might be some output pixels exceeding the range of [16, 235], so it's not the most general solution...

Answer 2

yuv to rgb conversion using FFMPEG I see lot of information given already for this above. However for code completeness I am re-sharing the code with missing allocPicture() function, header & library to include, it works for me like a charm. Thanks to @Валентин Никин & @Rotem for most of the info & code.

Headers:

#include <libswscale/swscale.h> 

Link FFMPEG Library:

libswscale

static AVFrame* allocPicture(enum AVPixelFormat pix_fmt, int width, int height)
{
  // Allocate a frame
  AVFrame* frame = av_frame_alloc();

  if (frame == NULL)
  {
    fprintf(stderr, "avcodec_alloc_frame failed");
  }

  if (av_image_alloc(frame->data, frame->linesize, width, height, pix_fmt, 1) < 0)
  {
    fprintf(stderr, "av_image_alloc failed");
  }

  frame->width = width;
  frame->height = height;
  frame->format = pix_fmt;

  return frame;
}

static AVFrame* avFrameConvertPixelFormat(const AVFrame* src, enum AVPixelFormat dstFormat)
{
  int width = src->width;
  int height = src->height;

  AVFrame* dst = allocPicture(dstFormat, width, height);

  struct SwsContext* conversion = sws_getContext(width,
                                        height,
                                        (enum AVPixelFormat)src->format,
                                        width,
                                        height,
                                        dstFormat,
                                        SWS_FAST_BILINEAR | SWS_FULL_CHR_H_INT | SWS_ACCURATE_RND,
                                        NULL,
                                        NULL,
                                        NULL);
  sws_scale(conversion, src->data, src->linesize, 0, height, dst->data, dst->linesize);
  sws_freeContext(conversion);

  dst->format = dstFormat;
  dst->width = src->width;
  dst->height = src->height;

  return dst;
}

// convert yuv420p10le to rgb24 (or any other RGB formats)
AVFrame* frame = avFrameConvertPixelFormat(frame, AV_PIX_FMT_RGB24);