how to save GridSearchCV xgboost model?

Question

I'm using xgboost to perform binary classification. I'm using GridSearchCV to find the best parameters. However, I don't know how to save the best model once the model with the best parameters has been discovered. How do I go about doing so?

Here is my code:

import xgboost as xgb
from sklearn.model_selection import StratifiedKFold, GridSearchCV

xgb_model = xgb.XGBClassifier(objective = "binary:logistic")

params = {
            'eta': np.arange(0.1, 0.26, 0.05),
            'min_child_weight': np.arange(1, 5, 0.5).tolist(),
            'gamma': [5],
            'subsample': np.arange(0.5, 1.0, 0.11).tolist(),
            'colsample_bytree': np.arange(0.5, 1.0, 0.11).tolist()
        }

scorers = {
            'f1_score':make_scorer(f1_score),
            'precision_score': make_scorer(precision_score),
            'recall_score': make_scorer(recall_score),
            'accuracy_score': make_scorer(accuracy_score)
          }

skf = StratifiedKFold(n_splits=10, shuffle = True)

grid = GridSearchCV(xgb_model, 
                    param_grid = params, 
                    scoring = scorers, 
                    n_jobs = -1, 
                    cv = skf.split(x_train, y_train),
                    refit = "accuracy_score")

grid.fit(x_train, y_train)
# Dictionary of best parameters
best_pars = grid.best_params_
# Save model
pickle.dump(grid.best_params_, open("xgb_log_reg.pickle", "wb"))

What I though the line that says # Save model would do was save the actual best param model. However, it just saves the dictionary best_pars. How do I go about saving the best model itself?

Answer 1

Try this:

# Dictionary of best parameters
best_pars = grid.best_params_
# Best XGB model that was found based on the metric score you specify
best_model = grid.best_estimator_
# Save model
pickle.dump(grid.best_estimator_, open("xgb_log_reg.pickle", "wb"))

You need [best_estimator_]

Link here