Lasso regression model has convergence warnings with GridSearchCV

Question

Here is the code that I have:

from sklearn.linear_model import Lasso
from sklearn.model_selection import GridSearchCV
import numpy as np

alpha_space = {'alpha': np.logspace(-4, 0, 50)}
lasso = Lasso(normalize=True,  tol=0.0001)
grid_search_lr = GridSearchCV (lasso, alpha_space, cv=3, scoring="neg_mean_squared_error")
grid_search_lr.fit(X_tr, y_tr)

print(grid_search_lr.best_params_)
print(np.sqrt(-grid_search_lr.best_score_))

But when I go to run it, I get at least 20 of these warnings before the answer:

/usr/local/lib/python3.7/dist-packages/sklearn/linear_model/_coordinate_descent.py:476: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 8451216620580.201, tolerance: 12888767617.309622

positive)

What should I do in order to fix these warnings or prevent them?

Answer 1

The Lasso estimator uses an iterative algorithm to solve the optimization problem. The iterative algorithm stops when it reaches the required level of convergence (set with the tolerance tol ). To avoid having the algorithm perform too many iterations (and possibly never stop), the algorithm also stops when it has performed a maximum number of iteration ( max_iter ). In this case, it raises a warning that it failed to reach the required level of convergence.

To avoid the convergence warning, you can either:

• increase the tolerance tol (to require a less strict convergence level)
• increase the maximum number of iteration max_iter (to spend more time finding the convergence level)