wav2vec2-base model 的 WER 在整个训练过程中保持为 1

Question

I am trying to run the wav2vec2 speech recognition model as shared in https://huggingface.co/docs/transformers/tasks/asr

This is the loss and WER during the training process, whereby the validation loss is reducing significantly, whereas the WER remains as 1.

I tried to print out the predicted and label values and this is what I got for the last 3 outputs , which results in the WER = 1.

This is the set of parameters of the model. model param .

What may actually go wrong here? Please help.. Thanks!

I have tried tuning the hyperparameters and hoping to reduce the WER.

Answer 1

Thank you for providing some useful information for troubleshooting.

• Your loss is reducing, which shows that the model is training, however your learning rate of 0.01 is very high. Consider changing this to something like 1e-5 as shown in the example on Hugging Face.

• The other thing I noticed was that all your input text is in UPPER CASE LIKE THIS . Depending on the training data used for the original model, it may not be expecting upper case text. Try lower-casing your text to see if that yields a lower WER.

• Your save_steps and eval_steps are also both far too high. This is how far the model "looks backwards" to evaluate - with a count of 1 on both these parameters, the model doesn't have enough history to compare better predictions. Increase these parameters and try again.