如何限制 Wav2Vec 中特征向量的大小？

Question

I'm attempting to receive a features vector of short wav (audio) files using wav2vec by using Hugging Face Transformers .

However, for unknown reasons, no matter which approach I use to control the output size, the results do not meet my requirements.

Ideally, I'd like to get all of the vectors to be the same length (eg 60K). I try to get it with the following command:

feature_extractor(input_audio, sampling_rate=16000, return_tensors="np", padding="max_length",
                                    max_length=60000).input_values

That command helped me create a minimal boundary of the data size by padding all the vectors into a minimum of 60K length, but I was surprised to see vectors with 120K values created as well.

Then I remove the padding parameter in the hope of obtaining vectors with no padding but an upper boundary of 60K. Based on the max_length documentation:

Maximum length of the returned list and optionally padding length

So I executed this line:

feature_extractor(input_audio, sampling_rate=16000, return_tensors="np",
                                    max_length=60000).input_values

Unexpectedly, I receive vectors ranging in length from 20K to 120K. Not limited at all.

---

To reproduce my bug and results, I've included a snippet of code and a link to relevant audio data.

import librosa
import numpy as np
from transformers import Wav2Vec2FeatureExtractor
from pathlib import Path

    p = Path(dataset_path)
    audio_files = [i.parents[0] / i.name for i in p.glob('**/*.wav')]
    feature_extractor = Wav2Vec2FeatureExtractor.from_pretrained('facebook/wav2vec2-base-960h')
    for file in (audio_files):
        input_audio, _ = librosa.load(file,
                                      sr=16000)
        features_with_padding = feature_extractor(input_audio, sampling_rate=16000,
                                return_tensors="np", padding="max_length", max_length=60000).input_values                                
        features_without_padding = feature_extractor(input_audio, sampling_rate=16000,
                                  return_tensors="np", max_length=60000).input_values
        print(features_with_padding.shape, features_without_padding.shape)

In this drive folder, I attached 2 wav files that create about 80K length vector.

How could I create a one-size feature vector with a wav2vec transformer?

Answer 1

At the moment truncation is not supported by the feature extractor in Hugging Face, so if you want to "pad" to a "max_length" that is shorter than the sample length, it simply won't change anything since no padding is needed.

However, we should definitely add a truncation functionality to Transformers as it is very important.