为 10 折交叉验证实验创建训练/测试 CSV 文件

Question

I have a CSV file (main.csv) that has a unique column ID that also pertains to my image names (minus their .jpg extension).

I want to do 10 fold cross-validation and create a train and test CSV's such that the test CSV for each fold would only contain 10 percent of the original CSV.

Is there a straightforward path (already done) to do this?

Basically, I want my eventual train and test CSV files to have the same exact column names but designed such that I could perform 10 fold cross validation with them (aka randomly sampled/shuffled and 10% selected).

I don't mind using pandas in Python or R.

I am not planning to use Scikit-learn for cross-validation as I am using my own manual code that is why I need the chopped train and test CSV for each of the folds.

Answer 1

Perhaps, you could be looking for this:

from sklearn.model_selection import train_test_split
#X contains the dependent columns from the CSV file, and Y is the predicted variable
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

test_size=0.33 This parameter says how much percentage of test data you want to split. All others would be train data.

X_train.to_csv(file_name, encoding='utf-8', index=False)

This code will save the X_train data to a CSV file which is 33%

y_train.to_csv(file_name, encoding='utf-8', index=False)

This code will save y_train data to CSV file which is 77%

This way, you can change the random_state value in the code each time and save file, so that you would get a good suffle. The number does not signify anything. It randomly shuffles and splits the dataset. (Perhaps, if we know the logic behind each number, it wont be a random split anymore !! :))

After this you can apply manual K fold.