Optuna for Catboost 以随机顺序输出“试验”？

Question

I'm working on hyperparameter tuning using Optuna for CatboostRegressor, however I realised that the trials I'm getting are in random order (mine started with Trial 7 and then Trial 5 then Trial 8. All of the examples I see online are in order, for example Trial 0 finished with value: xxxxx, Trial 1, Trial 2... (Example: https://www.kaggle.com/saurabhshahane/catboost-hyperparameter-tuning-with-optuna )

Is this an issue or it's not something to worry about? Not sure why is mine in a random order though.

Also wondering if I should be doing cb.cv (Catboost's cross validation) instead of cb.CatBoostRegressor and then .fit and .predict for hyperparameter tuning? Or it doesn't matter which way I'm using to get the best hyperparameters?

This is my code:

def objective(trial):

    optuna_params = {"subsample": trial.suggest_float("subsample", 0.5, 0.99),
                     'od_wait': trial.suggest_int('od_wait', 10, 50, step=1),
                     "colsample_bylevel": trial.suggest_float("colsample_bylevel", 0.5, 0.99),
                     "random_strength": trial.suggest_int("random_strength", 1, 10, step=1),
                     "l2_leaf_reg": trial.suggest_float("l2_leaf_reg", 1.0, 50.0),
                     "max_depth": trial.suggest_int("max_depth", 4, 10, step=1),
                     "n_estimators": trial.suggest_int("n_estimators", 100, 2500, step=1),
                     'learning_rate': trial.suggest_loguniform("learning_rate", 0.005, 0.1)}

    cbregressor = cb.CatBoostRegressor(**optuna_params, 
                                       random_state=0,
                                       loss_function='MAE', 
                                       eval_metric='MAE', 
                                       one_hot_max_size=0,
                                       boost_from_average=True)
    
    cat_optuna = cbregressor.fit(cat_train_pool2, eval_set=cat_val_pool2, verbose=False, early_stopping_rounds=10)
    
    y_valid_pred_cat3 = cat_optuna.predict(X_validation2)
    
    MAE = mean_absolute_error(y_validation, y_valid_pred_cat3)
    print('MAE score of CatBoost =', MAE)
    return MAE

study = optuna.create_study(direction="minimize", sampler = TPESampler(seed=0), study_name="Catboost Optuna")
study.optimize(objective, n_trials=100, n_jobs=-1)

Answer 1

Is this an issue or it's not something to worry about? Not sure why is mine in a random order though.

No. When we set n_jobs=-1 in study.optimize method, optimisation is performed parallelly using thread.

Regresser vs CV

I suppose either is fine. In general, when we use CV, overfitting is less likely to happen than single train/val split (ie, regresser in this setting). However, the computing of CV cost is expensive.