python-拆分数据集以获得高性能准确性的最佳技术

Question

I have applied these 4 methods:

1. Train and Test Sets.
2. K-fold Cross Validation.
3. Leave One Out Cross
4. Validation. Repeated Random Test-Train Splits.

The method "Train and Test Sets" achieve high accuracy but the remaining methods achieve same accuracy but lower then first approach.

I want to know which method should I choose?

Answer 1

Each of Train and Test Sets and Cross Validation used in certain case, Cross Validation used if you want to compare different models.Accuracy always increase if you use bigger training data that's why sometimes Leave One Out Cross perform better than K-fold Cross Validation ,it's depends on your dataset size and sometimes on algorithm you are using.On the other hand Train and Test Sets usually used if you aren't comparing diffrent models, and if the time requirements for running the cross validation aren't worth it,mean it's not needed to make Cross Validation in this case.In most cases Cross Validation is preferred,but, what method you should choose? this usually depend on your choices while training your data such way you handle data and algorithm such you are trainning data using Random Forests usually it's not needed to do Cross Validation but you can and do it in case need more you usually not doing Cross Validation in Random Forests when you use Out of Bag estimate .

Answer 2

Training a model comprises tuning model accuracy as well as model generalization. If model is not generalized it may be Underfit or Overfit model.

In this case, model may perform better on training data but accuracy may decrease on test or unknown data.

We use training data to improve the accuracy of model. As training data size increases model accuracy may also increase.

Similarly we use different training samples to generalize the model. So Train-Test splitting methods depend on the size of available data and algorithm used for model design.

First train-test method has a fix size training and testing data. So on each iteration, we use same train data to train model and same test data for model's accuracy assessment.

Second k-fold method has fix size train and test data but on each iteration, test and train data changes. So it may be a better approach irrespective of data size.

Leave one out approach is useful only if data size is small. Here we use almost whole data for training purpose. So training accuracy of model will be better but may not be a generalized model.

Randomised Train-test method is also a good approach for training and testing model's performance. Here we randomly select train and test data each time. So it may perform better than Leave one out method if data size is small.

And last each splitting approach has some pros and cons. So it depends on you which splitting method is good to your model. It also depends on data size and data selection means how we are selecting data from sample while splitting.