在深度学习中是否可以在训练集的子集上进行训练以找到最佳超参数？

Question

This question was migrated from Stack Overflow because it can be answered on Cross Validated. Migrated 7 days ago .

In classic machine learning, it is not uncommon to do a search for hyper-parameters by training different configurations on a small subset of training set. Usually, for each set of hyper-parameters, a k-fold cross validation is done over a small subset of training set. However, in deep learning, models are usually very hungry of data.

So, my question is that do you think is it still possible to use the same strategy in deep learning? What is your experience?

Answer 1

Yes, but as you noticed, the deep learning models usually work best with large samples. So your subset would need to be large as well. With insufficient data, the model would underperform and wouldn't help for hyperparameter tuning.