如何在 ffmpeg 的混音中插入某些其他声音时停止声音？

Question

I'm using a ffmpeg command that takes a set of sounds, mixes them into a single file, separating them by certain time intervals.

Below is how my command is today.

ffmpeg -i 
close_hh.wav    -i \
crash_l.wav     -i \
crash_r.wav     -i \
floor.wav       -i \
kick_l.wav      -i \
kick_r.wav      -i \
open_hh.wav     -i \
ride.wav        -i \
snare.wav       -i \
splash.wav      -i \
tom_1.wav       -i \
tom_2.wav       -i \
  tom_3.wav  -filter_complex  " [6]adelay=0|0[note_0];  [0]adelay=360|360[note_1];  [6]adelay=1260|1260[note_2];  [0]adelay=1537|1537[note_3];  [6]adelay=2494|2494[note_4];  [5]adelay=2767|2767[note_5];  [0]adelay=2969|2969[note_6];  [6]adelay=3673|3673[note_7];  [5]adelay=3924|3924[note_8];  [0]adelay=4132|4132[note_9];  [0][note_0][note_1][note_2][note_3][note_4][note_5][note_6][note_7][note_8][note_9]amix=inputs=11:normalize=0" record.wav

This is the resulting audio that this command generates:

ffmpg record.wav: https://drive.google.com/file/d/1LFV4ImLKLnRCqZRhZ7OqZy4Ecq5fwT3j/view?usp=sharing

The purpose is to generate a drum recording, so I would like to simulate the dynamics of the hi-hat sounds: When the closed hi-hat is played, the open hi-hat will stop playing immediately if it is still sounding. The same behavior does not happen for any of the other sounds.

One point that makes this a little more challenging is that other sounds can also be played between open hi-hat and closed hi-hat strikes, and theoretically the sound interruption behavior should work normally.

Below is a recording demonstrating the expected result. (My app already reproduces the sound result I need internally, so I just made a simple recording with the microphone to illustrate)

mic record.wav https://drive.google.com/file/d/19x19Fd_URQVo-MMCmGEHIC1SjaQbpWrh/view?usp=sharing

Notice that in the first audio (ffmpeg record.wav) the first sound (open hi-hat) continues playing after the second is played. In the second audio (mic record.wav) the first sound stops immediately after the second sound is played.

How should the ffmpeg command be to get the expected result?

Answer 1

While I'm not sure FFmpeg is the correct tool to synthesize music, here is a couple ways to approach your goal. First, I suggest to "compose" one instrument (in the case of a drumset, one drum/cymbal) at a time as illustrated below and then mix full tracks of instruments together.

Now, let's take the first 3 hi-hat segments to show how you can construct a filtergraph to form a hi-hat track.

(1) Trim each file to the length and concatenate all streams:

[0]atrim=duration=0.36[note_0]; \
[6]atrim=duration=0.9[note_1]; \
[0]atrim=duration=0.277[note_2]; \
[note_0][note_1][note_2]concat=3:0:1[aout]

While this works, if there is any audible sound at the trimmed moment, you'll likely hear the abrupt cut-off of the sound. This brings the alternate...

(2) Trim each file to the length + extra and cross-fade:

Let's say 10 ms overlap suffices, then you want to elongate each segment by 10 ms (0.01 s).

[0]atrim=duration=0.37[note_0]; \
[6]atrim=duration=0.91[note_1]; \
[0]atrim=duration=0.277[note_2]; \ # last file remain same length
[note_0][note_1]acrossfade=d=0.01:c1=exp:c2=exp[notes_01] \
[note_01][note_2]acrossfade=d=0.01:c1=exp:c2=exp[notes_012]

This will get ridiculously long, so you can shorten a bit by

[0]atrim=duration=0.37[note_0]; \
[6]atrim=duration=0.91,[note_0]acrossfade=d=0.01:c1=exp:c2=exp[notes_01]; \
[0]atrim=duration=0.277,[note_01]acrossfade=d=0.01:c1=exp:c2=exp[notes_012];

The c1 and c2 are the fading curve profiles. See this Wiki page for the available fade curves .

Finally, I arbitrarily chose 10 ms, and it is likely too short, so experiment and pick the best overlap duration.