使用 FFMPEG 将 arrays 的 stream 保存到视频

Question

我使用 CUDA 做了一个简单的流体模拟，我试图使用 FFMPEG 将它保存到视频中，但是我得到了Finishing stream 0:0 without any data written to it警告。

这就是我发送数据的方式

unsigned char* data = new unsigned char[SCR_WIDTH * SCR_HEIGHT * 4];
uchar4* pColors = new uchar4[SCR_WIDTH * SCR_HEIGHT];

for (int i = 0; i < N_FRAMES; i ++)
{
    // Computes a simulation step and sets pColors with the correct values.
    on_frame(pColors, timeStepSize);
    for (int j = 0; j < SCR_WIDTH * SCR_HEIGHT * 4; j+=4)
    {
        data[j] = pColors[j].x;
        data[j+1] = pColors[j].y;
        data[j+2] = pColors[j].z;
        data[j+3] = pColors[j].w;
    }
    std::cout.write(reinterpret_cast<char*>(data), SCR_WIDTH * SCR_HEIGHT * 4);
}

然后我使用以下命令将它传递给 FFMPEG：

./simulation.o | ffmpeg -y -f rawvideo -pixel_format rgba -video_size 1024x1024 -i - -c:v libx264 -pix_fmt yuv444p -crf 0 video.mp4 ./simulation.o | ffmpeg -y -f rawvideo -pixel_format rgba -video_size 1024x1024 -i - -c:v libx264 -pix_fmt yuv444p -crf 0 video.mp4

如果我对这些值进行硬编码（例如，如果我设置data[j] = 255 ，我会按预期得到一个红屏），这会很好用，但是当我使用pColors变量时，我会从 FFMPEG 收到以下消息

Finishing stream 0:0 without any data written to it.

即使pColors和data都持有正确的值。

这是来自 FFMPEG 的完整报告

ffmpeg started on 2022-12-13 at 14:28:34
Report written to "ffmpeg-20221213-142834.log"
Command line:
ffmpeg -y -f rawvideo -report -pixel_format rgba -video_size 128x128 -i - -c:v libx264 -pix_fmt yuv444p -crf 0 video9.mp4
ffmpeg version 3.4.11-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chrom  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Splitting the commandline.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rawvideo'.
Reading option '-report' ... matched as option 'report' (generate a report) with argument '1'.
Reading option '-pixel_format' ... matched as AVOption 'pixel_format' with argument 'rgba'.
Reading option '-video_size' ... matched as AVOption 'video_size' with argument '128x128'.
Reading option '-i' ... matched as input url with argument '-'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'libx264'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv444p'.
Reading option '-crf' ... matched as AVOption 'crf' with argument '0'.
Reading option 'video9.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option y (overwrite output files) with argument 1.
Applying option report (generate a report) with argument 1.
Successfully parsed a group of options.
Parsing a group of options: input url -.
Applying option f (force format) with argument rawvideo.
Successfully parsed a group of options.
Opening an input file: -.
[rawvideo @ 0x558eba7b0000] Opening 'pipe:' for reading
[pipe @ 0x558eba78a080] Setting default whitelist 'crypto'
[rawvideo @ 0x558eba7b0000] Before avformat_find_stream_info() pos: 0 bytes read:0 seeks:0 nb_streams:1
[rawvideo @ 0x558eba7b0000] After avformat_find_stream_info() pos: 0 bytes read:0 seeks:0 frames:0
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, bitrate: 13107 kb/s
    Stream #0:0, 0, 1/25: Video: rawvideo (RGBA / 0x41424752), rgba, 128x128, 13107 kb/s, 25 tbr, 25 tbn, 25 tbc
Successfully opened the file.
Parsing a group of options: output url video9.mp4.
Applying option c:v (codec name) with argument libx264.
Applying option pix_fmt (set pixel format) with argument yuv444p.
Successfully parsed a group of options.
Opening an output file: video9.mp4.
[file @ 0x558eba78a200] Setting default whitelist 'file,crypto'
Successfully opened the file.
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
cur_dts is invalid (this is harmless if it occurs once at the start per stream)
No more output streams to write to, finishing.
Finishing stream 0:0 without any data written to it.
detected 2 logical cores
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'video_size' to value '128x128'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'pix_fmt' to value '28'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'time_base' to value '1/25'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'pixel_aspect' to value '0/1'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'sws_param' to value 'flags=2'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] Setting 'frame_rate' to value '25/1'
[graph 0 input from stream 0:0 @ 0x558eba7a4a00] w:128 h:128 pixfmt:rgba tb:1/25 fr:25/1 sar:0/1 sws_param:flags=2
[format @ 0x558eba7a4b40] compat: called with args=[yuv444p]
[format @ 0x558eba7a4b40] Setting 'pix_fmts' to value 'yuv444p'
[auto_scaler_0 @ 0x558eba7a4be0] Setting 'flags' to value 'bicubic'
[auto_scaler_0 @ 0x558eba7a4be0] w:iw h:ih flags:'bicubic' interl:0
[format @ 0x558eba7a4b40] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[AVFilterGraph @ 0x558eba76d500] query_formats: 4 queried, 2 merged, 1 already done, 0 delayed
[auto_scaler_0 @ 0x558eba7a4be0] w:128 h:128 fmt:rgba sar:0/1 -> w:128 h:128 fmt:yuv444p sar:0/1 flags:0x4
[libx264 @ 0x558eba7cf900] using mv_range_thread = 24
[libx264 @ 0x558eba7cf900] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x558eba7cf900] profile High 4:4:4 Predictive, level 1.1, 4:4:4 8-bit
[libx264 @ 0x558eba7cf900] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=0 mixed_ref=1 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=0
Output #0, mp4, to 'video9.mp4':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0, 0, 1/12800: Video: h264 (libx264) (avc1 / 0x31637661), yuv444p, 128x128, q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.107.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (pipe:):
  Input stream #0:0 (video): 0 packets read (0 bytes); 0 frames decoded; 
  Total: 0 packets (0 bytes) demuxed
Output file #0 (video9.mp4):
  Output stream #0:0 (video): 0 frames encoded; 0 packets muxed (0 bytes); 
  Total: 0 packets (0 bytes) muxed
0 frames successfully decoded, 0 decoding errors
[AVIOContext @ 0x558eba7b4120] Statistics: 2 seeks, 3 writeouts
[AVIOContext @ 0x558eba7b4000] Statistics: 0 bytes read, 0 seeks

我以前从未使用过 FFMPEG，所以我很难发现我的错误。

Answer 1

我的猜测是simulation.o在到达std::cout.write之前因运行时错误而失败。

要调试问题，请执行：

./simulation.o > tmp.bin

检查tmp.bin的大小——大小应该是SCR_WIDTH*SCR_HEIGHT*N_FRAMES*4 。

---

为了测试，您可以尝试以下代码（我添加了_setmode(_fileno(stdout), O_BINARY)以实现 Windows 兼容性）：

#include <iostream>
#include <fstream>
#ifdef _MSC_VER
#include <fcntl.h>
#include <io.h>
#endif

#define SCR_WIDTH 128
#define SCR_HEIGHT 128
#define N_FRAMES 100

int main()
{
    unsigned char* data = new unsigned char[SCR_WIDTH * SCR_HEIGHT * 4];

#ifdef _MSC_VER
    //In Windows, we have to set stdout mode to binary mode (the default is text mode).
    //https://stackoverflow.com/a/16888382/4926757
    _setmode(_fileno(stdout), O_BINARY);
#endif


    for (int i = 0; i < N_FRAMES; i++)
    {
        // Computes a simulation step and sets pColors with the correct values.
        //on_frame(pColors, timeStepSize);
        for (int j = 0; j < SCR_WIDTH * SCR_HEIGHT * 4; j += 4)
        {
            //Fill with sum arbitrary data for testing.
            data[j] = (unsigned char)((j % (SCR_WIDTH*4)) * (j / (SCR_WIDTH*4))*10 + i*100); //pColors[j].x;
            data[j + 1] = (unsigned char)(((j % (SCR_WIDTH*4)) * (j / (SCR_WIDTH*4))*20 + i*101)); //pColors[j].y;
            data[j + 2] = (unsigned char)(((j % (SCR_WIDTH*4)) * (j / (SCR_WIDTH*4))*30 + i*102)); //pColors[j].z;
            data[j + 3] = (unsigned char)(((j % (SCR_WIDTH*4)) * (j / (SCR_WIDTH*4))*40 + i*103)); //pColors[j].w;
        }

        std::cout.write(reinterpret_cast<char*>(data), SCR_WIDTH * SCR_HEIGHT * 4);
    }

    delete[] data;

    return 0;
}

---

执行：

./simulation | ffmpeg -y -f rawvideo -report -pixel_format rgba -video_size 128x128 -i - -c:v libx264 -pix_fmt yuv444p -crf 0 video9.mp4

---

示例 output 帧：

笔记：
我在 Windows 10 中使用 FFmpeg 版本 5.1.2-full_build-www.gyan.dev 对其进行了测试。