[英]Creating a usable H.264 video file
I am trying to use libavcodec
to generate an mp4 video file from individual frames. 我正在尝试使用libavcodec
从各个帧生成mp4视频文件。 Each input frame is a qt QImage
, and the output file is written to using the Qt QFile
class. 每个输入帧都是一个qt QImage
,并且使用Qt QFile
类将输出文件写入。
I've done this through a VideoTarget
class which opens the given 'target' file when initialized, records frames when addFrame(image)
is called, and then saves/closes the file when its destructor is called. 我通过VideoTarget
类完成了此操作,该类在初始化时打开给定的“目标”文件,在调用addFrame(image)
时记录帧,然后在调用其析构函数时保存/关闭文件。
The class has the following fields: 该类具有以下字段:
AVCodec* m_codec = nullptr;
AVCodecContext *m_context = nullptr;
AVPacket* m_packet = nullptr;
AVFrame* m_frame = nullptr;
QFile m_target;
And looks like this: 看起来像这样:
VideoTarget::VideoTarget(QString target, QObject *parent) : QObject(parent), m_target(target)
{
// Find video codec
m_codec = avcodec_find_encoder_by_name("libx264rgb");
if (!m_codec) throw std::runtime_error("Unable to find codec.");
// Make codec context
m_context = avcodec_alloc_context3(m_codec);
if (!m_context) throw std::runtime_error("Unable to allocate codec context.");
// Make codec packet
m_packet = av_packet_alloc();
if (!m_packet) throw std::runtime_error("Unable to allocate packet.");
// Configure context
m_context->bit_rate = 400000;
m_context->width = 1280;
m_context->height = 720;
m_context->time_base = (AVRational){1, 60};
m_context->framerate = (AVRational){60, 1};
m_context->gop_size = 10;
m_context->max_b_frames = 1;
m_context->pix_fmt = AV_PIX_FMT_RGB24;
if (m_codec->id == AV_CODEC_ID_H264)
av_opt_set(m_context->priv_data, "preset", "slow", 0);
// Open Codec
int ret = avcodec_open2(m_context, m_codec, nullptr);
if (ret < 0) {
throw std::runtime_error("Unable to open codec.");
}
// Open file
if (!m_target.open(QIODevice::WriteOnly))
throw std::runtime_error("Unable to open target file.");
// Allocate frame
m_frame = av_frame_alloc();
if (!m_frame) throw std::runtime_error("Unable to allocate frame.");
m_frame->format = m_context->pix_fmt;
m_frame->width = m_context->width;
m_frame->height = m_context->height;
m_frame->pts = 0;
ret = av_frame_get_buffer(m_frame, 24);
if (ret < 0) throw std::runtime_error("Unable to allocate frame buffer.");
}
void VideoTarget::addFrame(QImage &image)
{
// Ensure frame data is writable
int ret = av_frame_make_writable(m_frame);
if (ret < 0) throw std::runtime_error("Unable to make frame writable.");
// Prepare image
for (int y = 0; y < m_context->height; y++) {
for (int x = 0; x < m_context->width; x++) {
auto pixel = image.pixelColor(x, y);
int pos = (y * 1024 + x) * 3;
m_frame->data[0][pos] = pixel.red();
m_frame->data[0][pos + 1] = pixel.green();
m_frame->data[0][pos + 2] = pixel.blue();
}
}
m_frame->pts++;
// Send the frame
ret = avcodec_send_frame(m_context, m_frame);
if (ret < 0) throw std::runtime_error("Unable to send AV frame.");
while (ret >= 0) {
ret = avcodec_receive_packet(m_context, m_packet);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
return;
else if (ret < 0) throw std::runtime_error("Error during encoding.");
m_target.write((const char*)m_packet->data, m_packet->size);
av_packet_unref(m_packet);
}
}
VideoTarget::~VideoTarget()
{
int ret = avcodec_send_frame(m_context, nullptr);
if (ret < 0) throw std::runtime_error("Unable to send AV null frame.");
while (ret >= 0) {
ret = avcodec_receive_packet(m_context, m_packet);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
return;
else if (ret < 0) throw std::runtime_error("Error during encoding.");
m_target.write((const char*)m_packet->data, m_packet->size);
av_packet_unref(m_packet);
}
// Magic number at the end of the file
uint8_t endcode[] = { 0, 0, 1, 0xb7 };
m_target.write((const char*)endcode, sizeof(endcode));
m_target.close();
// Free codec stuff
avcodec_free_context(&m_context);
av_frame_free(&m_frame);
av_packet_free(&m_packet);
}
When used, the class seems to work, and data is written to the file, except I am unable to play back the resulting file in any application. 使用该类时,该类似乎可以正常工作,并且数据已写入文件,但我无法在任何应用程序中播放生成的文件。
My main suspect is these lines: 我主要的怀疑是这些行:
// Prepare image
for (int y = 0; y < m_context->height; y++) {
for (int x = 0; x < m_context->width; x++) {
auto pixel = image.pixelColor(x, y);
int pos = (y * 1024 + x) * 3;
m_frame->data[0][pos] = pixel.red();
m_frame->data[0][pos + 1] = pixel.green();
m_frame->data[0][pos + 2] = pixel.blue();
}
}
The libavcodec
documentation was extremely vague regarding the layout of image data, so I effectively had to guess and be happy with the first thing that didn't crash, so chances are I'm writing this incorrectly. libavcodec
文档对于图像数据的布局非常含糊,因此我必须对没有崩溃的第一件事进行猜测并感到满意,因此可能是我写错了。 There's also the issue of size mismatch between my pixel
color data calls (giving int
values) and the 24-bits-per-pixel RGB format I have selected. 在我的pixel
颜色数据调用(给出int
值)和我选择的每像素24位RGB格式之间,还存在大小不匹配的问题。
How do I tweak this code to output actual, functioning video files? 如何调整此代码以输出实际有效的视频文件?
The libavcodec documentation was extremely vague regarding the layout of image data libavcodec文档对于图像数据的布局非常含糊
This is because every codec is different. 这是因为每个编解码器都不相同。 I recommend that you use yuv420p and not RGB24. 我建议您使用yuv420p而不是RGB24。 Many players can not play h264 rgb. 许多播放器无法播放h264 rgb。 You can use libswscale to convert between. 您可以使用libswscale进行转换。
Next, What format is the stream you are producing? 接下来,您制作的流是什么格式? Annex B can be played directly, but if you are using extradata + NALU size (AVCC), you will need to wrap the stream in a container. 附件B可以直接播放,但是如果您使用Extradata + NALU大小(AVCC),则需要将流包装在容器中。
Lastly, why use libavcodec? 最后,为什么要使用libavcodec? libx264 provides a cleaner API in my opinion. 我认为libx264提供了更简洁的API。 Unless you play to switch codecs later, avoid the abstraction. 除非稍后播放切换编解码器,否则请避免抽象。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.