简体繁体 English

使用pytorch训练神经网络时损失的周期性模式

[英]Periodic pattern in loss when training neural networks using pytorch

原文 2018-09-07 04:46:08 0 1 deep-learning/ pytorch/ mnist

I am using Pytorch train a model on MNIST, the loss curve has the periodic pattern shown in the figure. 我在MNIST上使用Pytorch训练模型，损耗曲线具有图中所示的周期性模式。 I've double checked the data loader and the dataset is shuffled in each epoch. 我仔细检查了数据加载器，并在每个时期对数据集进行了重新排序。 Any suggestions for the possible reason? 有任何建议可能原因吗？ Thanks. 谢谢。

Loss curve during training, training loss in blue and test loss in red 训练过程中的损失曲线，训练损失为蓝色，测试损失为红色

1 个解决方案

What I understand from the figure is that your loss is oscillating. 从图中我了解到，您的损失正在波动。 So try decreasing your learning rate and also some momentum term if available. 因此，请尝试降低学习率，并降低一些动力术语（如果有）。 I cannot guarantee you that this will work but hoping that it works it worth giving a try. 我不能保证这会起作用，但是希望它能起作用值得尝试。 Most of the things in Deep Learning can be explained, its just trial and error. 深度学习中的大多数事情都可以解释，只是反复试验。 Next time please ask such questions in https://ai.stackexchange.com or https://datascience.stackexchange.com . 下次，请在https://ai.stackexchange.com或https://datascience.stackexchange.com中提出此类问题。

在PyTorch中训练神经网络时，损失总是“为” - Loss is 'nan' all the time when training the neural network in PyTorch

使用 tensorflow 训练深度神经推荐模型时的 Nan Loss - Nan Loss when training Deep neural Recommender model using tensorflow

神经网络中的损失图 - Loss graph in neural networks

卷积神经网络中的损失函数 - Loss functions in Convolutional neural networks

jpeg压缩是否会影响使用卷积神经网络的训练和分类 - Will jpeg compression affect training and classification using Convolutional Neural Networks

在张量流教程中训练深度神经网络时的纳米损失 - nan loss when training a deep neural network in tensorflow tutorial

训练损失完全没有变化（PyTorch） - Training loss not changing at all (PyTorch)

Tensorflow：训练神经网络时损失没有改善 - Tensorflow: No improvement in loss while training neural net

使用神经网络提取模式 - Extracting Patterns using Neural Networks

如何使用正弦波和三角波生成图8模式（使用递归神经网络）？ - How would I generate a Figure 8 pattern (using recurrent neural networks) using a sine wave and triangular wave?

暂无

暂无

声明:本站的技术帖子网页，遵循CC BY-SA 4.0协议，如果您需要转载，请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.

相关问题 在PyTorch中训练神经网络时，损失总是“为” - Loss is 'nan' all the time when training the neural network in PyTorch 使用 tensorflow 训练深度神经推荐模型时的 Nan Loss - Nan Loss when training Deep neural Recommender model using tensorflow 神经网络中的损失图 - Loss graph in neural networks 卷积神经网络中的损失函数 - Loss functions in Convolutional neural networks jpeg压缩是否会影响使用卷积神经网络的训练和分类 - Will jpeg compression affect training and classification using Convolutional Neural Networks 在张量流教程中训练深度神经网络时的纳米损失 - nan loss when training a deep neural network in tensorflow tutorial 训练损失完全没有变化（PyTorch） - Training loss not changing at all (PyTorch) Tensorflow：训练神经网络时损失没有改善 - Tensorflow: No improvement in loss while training neural net 使用神经网络提取模式 - Extracting Patterns using Neural Networks 如何使用正弦波和三角波生成图8模式（使用递归神经网络）？ - How would I generate a Figure 8 pattern (using recurrent neural networks) using a sine wave and triangular wave?

相关标签

粤ICP备18138465号 © 2020-2024 STACKOOM.COM