XGBoost 超参数调整与具有多个类的 RandomizedSearchCV

Question

I am doing a hyperparameter tunning for my model and my code is like this:

para_tunning = {
        'learning_rate': [0.01,0.05,0.1],
        'min_child_weight': [1, 5, 10],
        'gamma': [0.5, 1, 1.5, 2, 5],
        'subsample': [0.6, 0.8, 1.0],
        'colsample_bytree': [0.6, 0.8, 1.0],
        'max_depth': [3, 4, 5, 6, 7, 8, 9, 10],
        "n_estimators": [100, 200, 300, 400, 500],
        "objective": "multi:softmax",
        "aplha":[0,2,4,6,8]
        }

clf_rndcv = RandomizedSearchCV(clf, 
                         param_distributions = para_tunning,
                         cv = 5,  
                         n_iter = 5,
                         scoring = 'accuracy', 
                         error_score = 0, 
                         verbose = 3, 
                         n_jobs = -1,
                         random_state = 42)
clf_rndcv.fit(X_train, y_train)

It shows that Fitting 5 folds for each of 5 candidates, totalling 25 fits , I suppose it just randomly pick 5 from the para_tunning dict and do a 5 fold cv? If I want to test all of the parameter do I switch over to gridsearchcv? And any suggestion for the tunning? I am doing a multiple class classifier with 100 classes, 500 sample per classes: so 50000 in total. Thanks!

Answer 1

Yes, if you want to search for ALL the hyperparameters you have to use GridSearchCV . GridSearch searches all possible combinations of the hyperparameters (it can be quite large given your case).

For your information, XGBoost has it own hyperparameters tuning. XGBoost CV