How can I plot a learning curve from a skopt/BayesSearchCV search

Question

I'm having trouble plotting a learning curve from a skopt optimization. Here is what I tried:

from skopt.space import Real, Integer, Categorical
from skopt.utils import use_named_args
from skopt import BayesSearchCV
from skopt.plots import plot_convergence

rf = RandomForestRegressor(random_state =7, n_jobs=4)
def RunSKOpt(X_train, y_train):  
    hyper_parameters =  {"n_estimators":      (5, 500),
                         "max_depth":         Categorical([3, None]),
                         "min_samples_split": (2, 10),
                         "min_samples_leaf":  (1, 10)
                        }

    search = BayesSearchCV(rf,
                           hyper_parameters,
                           n_iter = 40,
                           n_jobs = 4,
                           cv = 10,
                           verbose = 1,
                           return_train_score = False
    )
    return search

search = RunSKOpt(X_train, y_train)
search.fit(X_train, y_train)

plot_convergence(search)

The plot is empty. Please tell me what I'm doing wrong.

Charles

Answer 1

Directly from this Github Issue Thread: https://github.com/scikit-optimize/scikit-optimize/issues/751

BayesSearchCV was not intended for convergence plotting. You could however use the cv_results_ property of the *SearchCV, convert it to pandas (should be just creating dataframe out of the cv_results_ property) and then visualizing estimator performances for different iterations. The property is similar to those of GridSearchCV:

And here's an example of doing that: