如何使用ffmpeg缩短音频文件中的静音？

Question

I'm trying to shorten excess silence in audio recordings using ffmpeg (shorten them, not cutting silence out entirely). The current code I use:

ffmpeg -hide_banner -i file_name.m4a  -af silenceremove=0:0:0:-1:0.7:-30dB file_name_short.m4a

is not working. It detects silence longer than 0.7 seconds and remove them completely, which is not what I want. Anyone knows how to truncate silence, say, to shorten silence longer than 1 second down to 0.5 second?

Answer 1

The parameters of ffmpeg's silenceremove command seem to only allow you to delete all of the silence that is above a certain length. This means if you pass in stop_duration=0.5, and there is a block of silence that is 2.2 seconds long, you'll end up with 0.2 seconds of silence remaining (2.2 - 0.5 - 0.5 - 0.5 - 0.5 = 0.2).

If you don't mind converting back and forth between .wav format, you can use this Python script that I cooked up. It has quite a few options and even though it's in Python, it's using NumPy, so it can handle short files in much less than a second, and can handle a 2 hour long .wav in about 5.7 seconds, which is decent. For serious speed, this could be rewritten in C++. For videos, it may be possible to expand the solution using OpenCV.

Pluses:

• Can automatically determine threshold, with adjustable aggressiveness
• Can specify the maximum silence duration
• Can specify minimum non-silence duration to avoid momentary blips between silence
• Can use it just to detect periods of silence (much faster; processes 2 hours in 1.7 seconds)
• Avoids overwriting files unless told to do so

It's limited by the modules it uses. Catches are:

• it reads the whole file into memory
• it only works with wave files and doesn't keep the metadata. (see below for workaround)
• it can handle the common WAVE standards, unless you don't have SciPy installed, in which case it uses Python's wave module which only works well with 16-bit PCM

Usage in your case:

1. Convert m4a to wav: ffmpeg -i myfile.m4a myfile.wav
2. Run Silence Remover: python3 trim_silence.py --input=myfile.wav
3. Convert back with metadata: ffmpeg -i result.wav -i myfile.m4a -map_metadata 1 myfile_trimmed.m4a

Full usage notes:

usage: trim_silence.py [-h] --input INPUT [--output OUTPUT] [--threshold THRESHOLD] [--silence-dur SILENCE_DUR] [--non-silence-dur NON_SILENCE_DUR]
                             [--mode MODE] [--auto-threshold] [--auto-aggressiveness AUTO_AGGRESSIVENESS] [--detect-only] [--verbose] [--show-silence] [--time-it]
                             [--overwrite]

optional arguments:
  -h, --help            show this help message and exit
  --input INPUT         (REQUIRED) name of input wav file (default: None)
  --output OUTPUT       name of output wave file (default: result.wav)
  --threshold THRESHOLD
                        silence threshold - can be expressed in dB, e.g. --threshold=-25.5dB (default: -25dB)
  --silence-dur SILENCE_DUR
                        maximum silence duration desired in output (default: 0.5)
  --non-silence-dur NON_SILENCE_DUR
                        minimum non-silence duration between periods of silence of at least --silence-dur length (default: 0.1)
  --mode MODE           silence detection mode - can be 'any' or 'all' (default: all)
  --auto-threshold      automatically determine silence threshold (default: False)
  --auto-aggressiveness AUTO_AGGRESSIVENESS
                        aggressiveness of the auto-threshold algorithm. Integer between [-20,20] (default: 3)
  --detect-only         don't trim, just detect periods of silence (default: False)
  --verbose             print general information to the screen (default: False)
  --show-silence        print locations of silence (always true if --detect-only is used) (default: False)
  --time-it             show steps and time to complete them (default: False)
  --overwrite           overwrite existing output file, if applicable (default: False)

Contents of trim_silence.py :

import numpy as np
import argparse
import time
import sys
import os

def testmode(mode):
    mode = mode.lower()
    valid_modes = ["all","any"]
    if mode not in valid_modes:
        raise Exception("mode '{mode}' is not valid - must be one of {valid_modes}")
    return mode
def testaggr(aggr):
    try:
        aggr = min(20,max(-20,int(aggr)))
        return aggr
    except:
        raise Exception("auto-aggressiveness '{aggr}' is not valid - see usage")
    

parser = argparse.ArgumentParser(formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument("--input",                              type=str,   help="(REQUIRED) name of input wav file", required=True)
parser.add_argument("--output",       default="result.wav", type=str,   help="name of output wave file")
parser.add_argument("--threshold",    default="-25dB",      type=str,   help="silence threshold - can be expressed in dB, e.g. --threshold=-25.5dB")
parser.add_argument("--silence-dur",  default=0.5,          type=float, help="maximum silence duration desired in output")
parser.add_argument("--non-silence-dur", default=0.1,      type=float, help="minimum non-silence duration between periods of silence of at least --silence-dur length")
parser.add_argument("--mode",         default="all",        type=testmode,  help="silence detection mode - can be 'any' or 'all'")
parser.add_argument("--auto-threshold",action="store_true",             help="automatically determine silence threshold")
parser.add_argument("--auto-aggressiveness",default=3,type=testaggr, help="aggressiveness of the auto-threshold algorithm.  Integer between [-20,20]")
parser.add_argument("--detect-only",  action="store_true",              help="don't trim, just detect periods of silence")
parser.add_argument("--verbose",      action="store_true",              help="print general information to the screen")
parser.add_argument("--show-silence", action="store_true",              help="print locations of silence (always true if --detect-only is used)")
parser.add_argument("--time-it",      action="store_true",              help="show steps and time to complete them")
parser.add_argument("--overwrite",    action="store_true",              help="overwrite existing output file, if applicable")

args = parser.parse_args()
args.show_silence = args.show_silence or args.detect_only
if not args.detect_only and not args.overwrite:
    if os.path.isfile(args.output):
        print(f"Output file ({args.output}) already exists.  Use --overwrite to overwrite the existing file.")
        sys.exit(1)

if (args.silence_dur < 0):  raise Exception("Maximum silence duration must be >= 0.0")
if (args.non_silence_dur < 0):  raise Exception("Minimum non-silence duration must be >= 0.0")

try:
    from scipy.io import wavfile
    using_scipy = True
except:
    if args.verbose:  print("Failure using 'import scipy.io.wavfile'.  Using 'import wave' instead.")
    import wave
    using_scipy = False

if args.verbose: print(f"Inputs:\n  Input File: {args.input}\n  Output File: {args.output}\n  Max. Silence Duration: {args.silence_dur}\n  Min. Non-silence Duration: {args.non_silence_dur}")

from matplotlib import pyplot as plt
def plot(x):
    plt.figure()
    plt.plot(x,'o')
    plt.show()

def threshold_for_channel(ch):
    global data
    nbins = 100
    max_len = min(1024*1024*100,data.shape[0]) # limit to first 100 MiB
    if len(data.shape) > 1:
        x = np.abs(data[:max_len,ch]*1.0)
    else:
        x = np.abs(data[:max_len]*1.0)
    if data.dtype==np.uint8: x -= 127
    hist,edges = np.histogram(x,bins=nbins,density=True)
    slope = np.abs(hist[1:] - hist[:-1])
    argmax = np.argmax(slope < 0.00002)
    argmax = max(0,min(argmax + args.auto_aggressiveness, len(edges)-1))
    thresh = edges[argmax] + (127 if data.dtype==np.uint8 else 0)
    return thresh

def auto_threshold():
    global data
    max_thresh = 0
    channel_count = 1 if len(data.shape)==1 else data.shape[1]
    for ch in range(channel_count):
        max_thresh = max(max_thresh,threshold_for_channel(ch))
    return max_thresh
        

silence_threshold = str(args.threshold).lower().strip()
if args.auto_threshold:
    if args.verbose: print (f"  Silence Threshold: AUTO (aggressiveness={args.auto_aggressiveness})")
else:
    if "db" in silence_threshold:
        silence_threshold_db = float(silence_threshold.replace("db",""))
        silence_threshold = np.round(10**(silence_threshold_db/20.),6)
    else:
        silence_threshold = float(silence_threshold)
        silence_threshold_db = 20*np.log10(silence_threshold)

    if args.verbose: print (f"  Silence Threshold: {silence_threshold} ({np.round(silence_threshold_db,2)} dB)")
if args.verbose: print (f"  Silence Mode: {args.mode.upper()}")
if args.verbose: print("")
if args.time_it: print(f"Reading in data from {args.input}... ",end="",flush=True)
start = time.time()
if using_scipy:
    sample_rate, data = wavfile.read(args.input)
    input_dtype = data.dtype
    Ts = 1./sample_rate
    
    if args.auto_threshold:
        silence_threshold = auto_threshold()
    else:
        if data.dtype != np.float32:
            sampwidth = data.dtype.itemsize
            if (data.dtype==np.uint8):  silence_threshold += 0.5 # 8-bit unsigned PCM
            scale_factor = (256**sampwidth)/2.
            silence_threshold *= scale_factor
else:
    handled_sampwidths = [2]
    with wave.open(args.input,"rb") as wavin:
        params = wavin.getparams()
        if params.sampwidth in handled_sampwidths:
            raw_data = wavin.readframes(params.nframes)
    if params.sampwidth not in handled_sampwidths:
        print(f"Unable to handle a sample width of {params.sampwidth}")
        sys.exit(1)
end = time.time()
if args.time_it: print(f"complete (took {np.round(end-start,6)} seconds)")

if not using_scipy:
    if args.time_it: print(f"Unpacking data... ",end="",flush=True)
    start = time.time()
    Ts = 1.0/params.framerate
    if params.sampwidth==2: # 16-bit PCM
        format_ = 'h'
        data = np.frombuffer(raw_data,dtype=np.int16)
    elif params.sampwidth==3: # 24-bit PCM
        format_ = 'i'
        print(len(raw_data))
        data = np.frombuffer(raw_data,dtype=np.int32)
    
    data = data.reshape(-1,params.nchannels) # reshape into channels
    if args.auto_threshold:
        silence_threshold = auto_threshold()
    else:
        scale_factor = (256**params.sampwidth)/2. # scale to [-1:1)
        silence_threshold *= scale_factor
    data = 1.0*data # convert to np.float64
    end = time.time()
    if args.time_it: print(f"complete (took {np.round(end-start,6)} seconds)")

silence_duration_samples = args.silence_dur / Ts

if args.verbose: print(f"Input File Duration = {np.round(data.shape[0]*Ts,6)}\n")

combined_channel_silences = None
def detect_silence_in_channels():
    global combined_channel_silences
    if len(data.shape) > 1:
        if args.mode=="any":
            combined_channel_silences = np.min(np.abs(data),axis=1) <= silence_threshold
        else:
            combined_channel_silences = np.max(np.abs(data),axis=1) <= silence_threshold
    else:
        combined_channel_silences = np.abs(data) <= silence_threshold

    combined_channel_silences = np.pad(combined_channel_silences, pad_width=1,mode='constant',constant_values=0)
    
    
def get_silence_locations():
    global combined_channel_silences

    starts =  combined_channel_silences[1:] & ~combined_channel_silences[0:-1]
    ends   = ~combined_channel_silences[1:] &  combined_channel_silences[0:-1]
    start_locs = np.nonzero(starts)[0]
    end_locs   = np.nonzero(ends)[0]
    durations  = end_locs - start_locs
    long_durations = (durations > silence_duration_samples)
    long_duration_indexes = np.nonzero(long_durations)[0]
    
    if len(long_duration_indexes) > 1:
        non_silence_gaps = start_locs[long_duration_indexes[1:]] - end_locs[long_duration_indexes[:-1]]
        short_non_silence_gap_locs = np.nonzero(non_silence_gaps <= (args.non_silence_dur/Ts))[0]
        for loc in short_non_silence_gap_locs:
            if args.verbose and args.show_silence:
                ns_gap_start = end_locs[long_duration_indexes[loc]] * Ts
                ns_gap_end   = start_locs[long_duration_indexes[loc+1]] * Ts
                ns_gap_dur   = ns_gap_end - ns_gap_start
                print(f"Removing non-silence gap at {np.round(ns_gap_start,6)} seconds with duration {np.round(ns_gap_dur,6)} seconds")
            end_locs[long_duration_indexes[loc]] = end_locs[long_duration_indexes[loc+1]]
                
        long_duration_indexes = np.delete(long_duration_indexes, short_non_silence_gap_locs + 1)

    if args.show_silence:
        if len(long_duration_indexes)==0:
            if args.verbose: print("No periods of silence found")
        else:
            if args.verbose: print("Periods of silence shown below")
            fmt_str = "%-12s  %-12s  %-12s"
            print(fmt_str % ("start","end","duration"))
            for idx in long_duration_indexes:
                start = start_locs[idx]
                end = end_locs[idx]
                duration = end - start
                print(fmt_str % (np.round(start*Ts,6),np.round(end*Ts,6),np.round(duration*Ts,6)))
        if args.verbose: print("")

    return start_locs[long_duration_indexes], end_locs[long_duration_indexes]

def trim_data(start_locs,end_locs):
    global data
    if len(start_locs)==0: return
    keep_at_start = int(silence_duration_samples / 2)
    keep_at_end = int(silence_duration_samples - keep_at_start)
    start_locs = start_locs + keep_at_start
    end_locs = end_locs - keep_at_end
    delete_locs = np.concatenate([np.arange(start_locs[idx],end_locs[idx]) for idx in range(len(start_locs))])
    data = np.delete(data, delete_locs, axis=0)

def output_data(start_locs,end_locs):
    global data
    if args.verbose: print(f"Output File Duration = {np.round(data.shape[0]*Ts,6)}\n")
    if args.time_it: print(f"Writing out data to {args.output}... ",end="",flush=True)
    if using_scipy:
        wavfile.write(args.output, sample_rate, data)
    else:
        packed_buf = data.astype(format_).tobytes()
        with wave.open(args.output,"wb") as wavout:
            wavout.setparams(params) # same params as input
            wavout.writeframes(packed_buf)

start = time.time()
if not args.verbose and args.time_it: print("Detecting silence... ",end="",flush=True)
detect_silence_in_channels()
(start_locs,end_locs) = get_silence_locations()
end = time.time()
if not args.verbose and args.time_it: print(f"complete (took {np.round(end-start,6)} seconds)")

if args.detect_only:
    if args.verbose: print("Not trimming, because 'detect only' flag was set")
else:
    if args.time_it: print("Trimming data... ",end="",flush=True)
    start = time.time()
    trim_data(start_locs,end_locs)
    end = time.time()
    if args.time_it: print(f"complete (took {np.round(end-start,6)} seconds)")
    start = time.time()
    output_data(start_locs, end_locs)
    end = time.time()
    if args.time_it: print(f"complete (took {np.round(end-start,6)} seconds)")

If you want a script that assumes 16-bit PCM and without all the extra print statements and what not:

import numpy as np
from scipy.io import wavfile

# Params
(infile,outfile,threshold_db,silence_dur,non_silence_dur,mode) = ("test_stereo.wav","result.wav",-25,0.5,0.1,"all")
silence_threshold = np.round(10**(threshold_db/20.),6) * 32768 # Convert from dB to linear units and scale, assuming 16-bit PCM input

# Read data
Fs, data = wavfile.read(infile)
silence_duration_samples = silence_dur * Fs
if len(data.shape)==1: data = np.expand_dims(data,axis=1)

# Find silence
find_func = np.min if mode=="any" else np.max
combined_channel_silences = find_func(np.abs(data),axis=1) <= silence_threshold
combined_channel_silences = np.pad(combined_channel_silences, pad_width=1,mode='constant',constant_values=0)

# Get start and stop locations
starts =  combined_channel_silences[1:] & ~combined_channel_silences[0:-1]
ends   = ~combined_channel_silences[1:] &  combined_channel_silences[0:-1]
start_locs = np.nonzero(starts)[0]
end_locs   = np.nonzero(ends)[0]
durations  = end_locs - start_locs
long_durations = (durations > silence_duration_samples)
long_duration_indexes = np.nonzero(long_durations)[0]
    
# Cut out short non-silence between silence
if len(long_duration_indexes) > 1:
    non_silence_gaps = start_locs[long_duration_indexes[1:]] - end_locs[long_duration_indexes[:-1]]
    short_non_silence_gap_locs = np.nonzero(non_silence_gaps <= (non_silence_dur * Fs))[0]
    for loc in short_non_silence_gap_locs:
        end_locs[long_duration_indexes[loc]] = end_locs[long_duration_indexes[loc+1]]
    long_duration_indexes = np.delete(long_duration_indexes, short_non_silence_gap_locs + 1)

    (start_locs,end_locs) = (start_locs[long_duration_indexes], end_locs[long_duration_indexes])

# Trim data
if len(long_duration_indexes) > 1:
    if len(start_locs) > 0:
        keep_at_start = int(silence_duration_samples / 2)
        keep_at_end = int(silence_duration_samples - keep_at_start)
        start_locs = start_locs + keep_at_start
        end_locs = end_locs - keep_at_end
        delete_locs = np.concatenate([np.arange(start_locs[idx],end_locs[idx]) for idx in range(len(start_locs))])
        data = np.delete(data, delete_locs, axis=0)

# Output data
wavfile.write(outfile, Fs, data)