如何从循环通过多个 alpha 的套索回归中提取给我最低均方误差的 alpha？

Question

I am attempting to find the optimal combination of x values, their respective exponent's, and the alpha that will allow me to find the lowest mean squared error.

I used a Lasso regression from SKlearn, but so far I am only able to determine the minimum MSE, and the variable combination that creates it. I am not sure how to pull the alpha out that allowed it, or how to see if the combinations of variables have any exponents associated with them.

The results I achieved:

outcomes from the Best Lasso Regression Model: minimum Avg Test MSE: 9172.38 The Combination of Variables: ['Date', 'Cargo_size', 'Parcel_size', 'Rest', 'Sub']

x_combos = []
for n in range(1,9):
    combos = combinations(['Date', 'Cargo_size', 'Parcel_size', 'Rest', 'Age',\
                                           'Sub', 'X_coord', 'Y_coord'], n)
    x_combos.extend(combos)

lasso_models = {}
alphas = 10**np.linspace(10,-2, 100)*.5    
   
for n in range(0, len(x_combos)):
        combo_list = list(x_combos[n])
        x = data[combo_list]
        poly = PolynomialFeatures(3)
        poly_x = poly.fit_transform(x)
        model = Lasso(max_iter=100000, normalize=(True))
        for a in alphas:
            model.set_params(alpha = a)
            model.fit(poly_x,y) #
        lasso_cv_scores = cross_validate(model, poly_x, y, cv=10, scoring=('neg_mean_squared_error', 'r2'), return_train_score=(True), return_estimator=(True))
        lasso_models[str(combo_list)] = np.mean(lasso_cv_scores['test_neg_mean_squared_error'])
    
    
    
    
    print("outcomes from the Best Lasso Regression Model:")
    min_mse = abs(max(lasso_models.values()))
    print("minimum Avg Test MSE:", min_mse.round(2))
    for possibles, i in lasso_models.items():
        if i == -min_mse:
            print("The Combination of Variables:", possibles)

Answer 1

You can perform this operation with GridSearchCV which is an object that perform an exhaustive search over specified parameter values for an estimator.

As follows:

from sklearn.model_selection import GridSearchCV
param_grid = { 'alpha' : alphas}
cv = GridSearchCV(model,
                  n_jobs=-1,
                  param_grid=param_grid,
                  cv=5,return_train_score=(True)
                 ).fit(poly_x, y_train)
cv.best_params_

This snippet will perfrom an hyperparameter search over the alphas and return the best set of parameters. Additionally you can return the best fitted model using cv.best_estimator_ .