FFmpeg v4l2 復制不穩定

Question

序幕

我在 Ubuntu 20.04 機器上使用ffmpeg 4.2.2 克隆 USB 視頻網絡攝像頭的源（ v4l2應用程序設備可以作為/dev/video0訪問）。 我使用了這個ffmpeg命令，效果很好：

ffmpeg -f v4l2 -i /dev/video0 -codec copy -f v4l2 /dev/video1

本質上，這使用/dev/video1形式的v4l2環回設備（虛擬設備），而消費應用程序只是將其視為常規v4l2設備。

之后效果很好，我遇到了更改 output 像素格式的需要（相關的SO question ）。 我的源設備提供yuv420p而接收器應用程序只能處理yuyv422 。 為此，我使用-pix_fmt參數。 但是，使用-codec copy時不支持此功能，因此我需要明確指定編解碼器。 這沒問題，因為v4l2設備僅支持原始視頻（至少據我所知）。 因此，我最終得到了這個：

ffmpeg -f v4l2 -i /dev/video0 -c:v rawvideo -pix_fmt yuyv422 -f v4l2 /dev/video1

問題

不幸的是，這似乎非常不穩定。 在啟動此命令的 10 次中大約有 1 次按預期工作： ffmpeg開始克隆提要並轉換顏色格式：

joel@joel-ubuntu:~$ ffmpeg -f v4l2 -i /dev/video0 -c:v rawvideo -pix_fmt yuyv422 -f v4l2 /dev/video1
ffmpeg version 4.2.2-1ubuntu1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-3ubuntu1)
  configuration: --prefix=/usr --extra-version=1ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[video4linux2,v4l2 @ 0x55a4f38a5700] Time per frame unknown
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 11224.864514, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480, 14.33 tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, video4linux2,v4l2, to '/dev/video1':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 640x480, q=2-31, 70451 kb/s, 14.33 fps, 14.33 tbn, 14.33 tbc
    Metadata:
      encoder         : Lavc58.54.100 rawvideo
frame=  214 fps= 15 q=-0.0 Lsize=N/A time=00:00:14.93 bitrate=N/A dup=1 drop=0 speed=1.05x    
video:128400kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

但是，其他時候我會遇到非常過時的行為，其中 output 除了第一幀（然后它凍結）之外永遠無法使用，並且ffmpeg在短短幾秒鍾內提到超過 100k 重復幀以及報告 FPS 超過 30' 000。 幾秒鍾后它會崩潰，或者它只是掛起，直到我Ctrl^C它：

joel@joel-ubuntu:~$ ffmpeg -f v4l2 -i /dev/video0 -c:v rawvideo -pix_fmt yuyv422 -f v4l2 /dev/video1
ffmpeg version 4.2.2-1ubuntu1 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-3ubuntu1)
  configuration: --prefix=/usr --extra-version=1ubuntu1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[video4linux2,v4l2 @ 0x561ef693c700] Time per frame unknown
Input #0, video4linux2,v4l2, from '/dev/video0':
  Duration: N/A, start: 11156.178415, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480, 1000k tbr, 1000k tbn, 1000k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[video4linux2,v4l2 @ 0x561ef6940400] Frame rate very high for a muxer not efficiently supporting it.
Please consider specifying a lower framerate, a different muxer or -vsync 2
Output #0, video4linux2,v4l2, to '/dev/video1':
  Metadata:
    encoder         : Lavf58.29.100
    Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 640x480, q=2-31, 4915200000 kb/s, 1000k fps, 1000k tbn, 1000k tbc
    Metadata:
      encoder         : Lavc58.54.100 rawvideo
More than 1000 frames duplicated
More than 10000 frames duplicated
More than 100000 frames duplicated=N/A time=00:00:00.15 bitrate=N/A dup=151050 drop=0 speed=0.0456x    
27653321 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=0 speed=0.0289x    
27733577 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=1 speed=0.0256x    
27787315 frame duplication too large, skipping
27859335 frame duplication too large, skipping
27941595 frame duplication too large, skipping
28006927 frame duplication too large, skipping
28069615 frame duplication too large, skipping
28140781 frame duplication too large, skipping
28217871 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=8 speed=0.0251x    
28282039 frame duplication too large, skipping
28347777 frame duplication too large, skipping
28449099 frame duplication too large, skipping
28491015 frame duplication too large, skipping
28565585 frame duplication too large, skipping
28630457 frame duplication too large, skipping
28710537 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=15 speed=0.0247x    
28770549 frame duplication too large, skipping
28843129 frame duplication too large, skipping
28922285 frame duplication too large, skipping
28985573 frame duplication too large, skipping
29061631 frame duplication too large, skipping
29133801 frame duplication too large, skipping
29193197 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=22 speed=0.0243x    
29269443 frame duplication too large, skipping
29349681 frame duplication too large, skipping
29403629 frame duplication too large, skipping
29489299 frame duplication too large, skipping
29542137 frame duplication too large, skipping
29611859 frame duplication too large, skipping
29688271 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=29 speed=0.0238x    
29759777 frame duplication too large, skipping
29824489 frame duplication too large, skipping
29897671 frame duplication too large, skipping
29966443 frame duplication too large, skipping
30037557 frame duplication too large, skipping
30117317 frame duplication too large, skipping
30172847 frame duplication too large, skipping
30255657 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=37 speed=0.0234x    
30331325 frame duplication too large, skipping
30385245 frame duplication too large, skipping
30458209 frame duplication too large, skipping
30532393 frame duplication too large, skipping
30596453 frame duplication too large, skipping
30681671 frame duplication too large, skipping
30740757 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=44 speed=0.023x    
30806259 frame duplication too large, skipping
30873657 frame duplication too large, skipping
30950843 frame duplication too large, skipping
31013967 frame duplication too large, skipping
31099321 frame duplication too large, skipping
31172533 frame duplication too large, skipping
31224713 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=51 speed=0.0227x    
31297053 frame duplication too large, skipping
31378711 frame duplication too large, skipping
31438565 frame duplication too large, skipping
31506345 frame duplication too large, skipping
31589213 frame duplication too large, skipping
31656175 frame duplication too large, skipping
31718295 frame duplication too large, skipping
31814019 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=59 speed=0.0223x    
31876469 frame duplication too large, skipping
31917425 frame duplication too large, skipping
31995521 frame duplication too large, skipping
32062573 frame duplication too large, skipping
32127183 frame duplication too large, skipping
32197109 frame duplication too large, skipping
32267481 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=66 speed=0.022x    
32358605 frame duplication too large, skipping
32407853 frame duplication too large, skipping
32478077 frame duplication too large, skipping
32523429 frame duplication too large, skipping
32557449 frame duplication too large, skipping
32587093 frame duplication too large, skipping
32620687 frame duplication too large, skipping
32654747 frame duplication too large, skipping
32689485 frame duplication too large, skipping
32719435 frame duplication too large, skipping
32753423 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=77 speed=0.0216x    
32781827 frame duplication too large, skipping
32820747 frame duplication too large, skipping
32860855 frame duplication too large, skipping
32884671 frame duplication too large, skipping
32918767 frame duplication too large, skipping
32955299 frame duplication too large, skipping
32989505 frame duplication too large, skipping
33019757 frame duplication too large, skipping
33053493 frame duplication too large, skipping
33082955 frame duplication too large, skipping
33116965 frame duplication too large, skipping
33156613 frame duplication too large, skipping
33188409 frame duplication too large, skipping
33224915 frame duplication too large, skipping
33255715 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=92 speed=0.0213x    
33292845 frame duplication too large, skipping
33323677 frame duplication too large, skipping
33352059 frame duplication too large, skipping
33386205 frame duplication too large, skipping
33420195 frame duplication too large, skipping
33455171 frame duplication too large, skipping
33484311 frame duplication too large, skipping
33520791 frame duplication too large, skipping
33550153 frame duplication too large, skipping
33587191 frame duplication too large, skipping
33623055 frame duplication too large, skipping
33656991 frame duplication too large, skipping
33686655 frame duplication too large, skipping
33720711 frame duplication too large, skipping
33749987 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=107 speed=0.021x    
33784387 frame duplication too large, skipping
33823663 frame duplication too large, skipping
33869275 frame duplication too large, skipping
33907671 frame duplication too large, skipping
34043059 frame duplication too large, skipping
34089679 frame duplication too large, skipping
34120031 frame duplication too large, skipping
34193323 frame duplication too large, skipping
34263479 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=116 speed=0.0207x    
34403751 frame duplication too large, skipping
34470319 frame duplication too large, skipping
34542507 frame duplication too large, skipping
34613587 frame duplication too large, skipping
34687495 frame duplication too large, skipping
34755115 frame duplication too large, skipping:00:00.70 bitrate=N/A dup=707630 drop=122 speed=0.0204x    
34823847 frame duplication too large, skipping
34893931 frame duplication too large, skipping
34963903 frame duplication too large, skipping
35034011 frame duplication too large, skipping
35103715 frame duplication too large, skipping
35178691 frame duplication too large, skipping
frame=707641 fps=20090 q=-0.0 Lsize=N/A time=00:00:00.70 bitrate=N/A dup=707630 drop=129 speed=0.0201x    
video:424584600kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Exiting normally, received signal 2.

我不知道是什么導致了這種行為，也不知道如何追蹤和修復它。 是否有人對這種行為有經驗，或者我的設置是否存在簡單問題？

注意：我正在使用 DroidCam。 DroidCam 提供了一個 Linux 客戶端，該客戶端又提供了一個v4l2 output 設備。 除了一些小問題，我沒有注意到任何與普通網絡攝像頭不同的行為。 不幸的是，我無法使用普通的 USB UVC 網絡攝像頭重現這一點，因為網絡攝像頭已經提供了所需的 output 像素格式。 這種 output 像素格式轉換確實是必要的，因為DroidCam輸出yuv420p但使用 stream 的應用程序只能處理yuyv422 。

Answer 1

V4L2 設備是恆定幀率寫入器，因此如果源幀率較低，ffmpeg 將復制幀以保持恆定幀率。 在沒有用戶設置速率的情況下，ffmpeg 將其設置為輸入的幀速率。 對於未明確發出幀速率信號的捕獲設備，ffmpeg 通過查找幀之間的最小間隔來推斷速率。

當 ffmpeg 檢測到1000k tbr時，就會出現大量重復。 設置一個合理的數字-r 15或禁用幀復制-vsync vfr 。