在ffmpeg中将其他音频文件与视频（+音频）混合使用

Question

I'm trying to mix additional audio file with video which has also audio within. But the problem is that I already have complex ffmpeg command and don't know how to combine them together.

This is my existing ffmpeg command which uses some offsets and replaces additional audio file with embedded one (audio inside video) and also overlays few gauges and watermark to the video.

ffmpeg -y 
-ss 00:00:01:213 -i videoFile.mp4 
-ss 00:00:03:435 -i audioFile.wav 
-i watermark.png 
-framerate 6 -i gauge1_path/img-%04d.png 
-framerate 1 -i gauge2_path/img-%04d.png 
-framerate 2 -i gauge3_path/img-%04d.png 
-framerate 2 -i gauge4_path/img-%04d.png 
-framerate 2 -i gauge5_path/img-%04d.png 
-framerate 2 -i gauge6_path/img-%04d.png 
-filter_complex [0][2]overlay=(21):(H-h-21)[ovr0];
[ovr0][3]overlay=(W-w-21):(H-h-21)[ovr1];
[ovr1][4]overlay=(W-w-21):(H-h-333)[ovr2];
[ovr2][5]overlay=(W-w-21):(H-h-418)[ovr3];
[ovr3][6]overlay=(W-w-21):(H-h-503)[ovr4];
[ovr4][7]overlay=(W-w-21):(H-h-588)[ovr5];
[ovr5][8]overlay=(W-w-21):(H-h-673)
-map 0:v -map 1:a -c:v libx264 -preset ultrafast -crf 23 -t 00:5:10:000 output.mp4

Now I would like to use ffmpeg's amix in order to mix both audios instead of replacing them, if possible with ability to set volumes. But official documentation amix says nothing about volume.

Separately both seems to work ok.

ffmpeg -y -i video.mp4 -i audio.mp3 -filter_complex [0][1]amix=inputs=2[a] -map 0:v -map [a] -c:v copy output.mp4

and

ffmpeg -y -i video.mp4 -i audio.mp3 -i watermark.png -filter_complex [0][2]overlay=(21):(H-h-21)[ovr0] -map [ovr0]:v -map 1:a -c:v libx264 -preset ultrafast -crf 23 output.mp4

but together

ffmpeg -y -i video.mp4 -i audio.mp3 -i watermark.png -filter_complex [0][1]amix=inputs=2[a];[a][2]overlay=(21):(H-h-21)[ovr0] -map [ovr0]:v -map [a] -c:v libx264 -preset ultrafast -crf 23 output.mp4

I'm getting an error:

ffmpeg version N-93886-gfbdb3aa179 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.3.1 (GCC) 20190414
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 28.100 / 56. 28.100
  libavcodec     58. 52.101 / 58. 52.101
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 53.101 /  7. 53.101
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 1970-01-01T00:00:00.000000Z
    encoder         : Lavf53.24.2
  Duration: 00:00:29.57, start: 0.000000, bitrate: 1421 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1032 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
    Metadata:
      creation_time   : 1970-01-01T00:00:00.000000Z
      handler_name    : SoundHandler
[mp3 @ 0000015e2f934ec0] Estimating duration from bitrate, this may be inaccurate
Input #1, mp3, from 'audio.mp3':
  Duration: 00:00:45.33, start: 0.000000, bitrate: 128 kb/s
    Stream #1:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s
Input #2, png_pipe, from 'watermark.png':
  Duration: N/A, bitrate: N/A
    Stream #2:0: Video: png, rgb24(pc), 100x56 [SAR 3779:3779 DAR 25:14], 25 tbr, 25 tbn, 25 tbc
[Parsed_amix_0 @ 0000015e2ff2e940] Media type mismatch between the 'Parsed_amix_0' filter output pad 0 (audio) and the 'Parsed_overlay_1' filter input pad 0 (video)
[AVFilterGraph @ 0000015e2f91c600] Cannot create the link amix:0 -> overlay:0
Error initializing complex filters.
Invalid argument

So my question: whether it's possible to combine amix and overlay together and how and in which order they should be used? Or should I look something different because amix unable to set volume levels?

Thanks in advance!

Answer 1

Use

ffmpeg -y -i video.mp4 -i audio.mp3 -i watermark.png -filter_complex [0][1]amix=inputs=2,volume=2[a];[0][2]overlay=(21):(Hh-21)[ovr0] -map [ovr0]:v -map [a] -c:v libx264 -preset ultrafast -crf 23 output.mp4

You're sending the mixed audio to the overlay filter which requires video input. Overlay should be fed the original video stream. The audio output [a] should left alone. It is consumed as a mapped output stream.

volume filter added after amix to restore volume. amix reduces volume, in order to avoid clipping.