Why RandomizedSearchCV reeturn a value of degree or gamma for linear estimator?

Question

With Sklearn I am using RandomizedSearchCV, in a specific case the best estimator is:

SVR(C=1594.0828461797396, degree=0.8284528822863231, gamma=1.1891370222133257,kernel='linear')

But according to sklearn documentation , degree and gamma is just for rbf and poly kernels. Why I get linar estimator with gamma and degree values?

from sklearn.model_selection import RandomizedSearchCV
from scipy.stats import expon, reciprocal

param_distribs = {
        'kernel': ['linear', 'rbf','poly','sigmoid'],
        'C': reciprocal(20, 200000),
        'gamma': expon(scale=1.0),
        'degree': expon(scale=1.0),
    }

svm_reg = SVR()
rnd_search = RandomizedSearchCV(svm_reg, param_distributions=param_distribs,
                                n_iter=50, cv=5, scoring='neg_mean_squared_error',
                                verbose=2, random_state=42)
rnd_search.fit(X, y)

Answer 1

RandomizedSearchCV will always randomly set all specified parameters for the estimator irrespective of such restrictions as there is no internal method implemented to check which combinations make sense for a particular estimator or not. Since gamma and degree are just ignored in combination with a linear kernel, it will also not raise an error and the algorithm just runs with all parameters set each time.

If you want to avoid such behavior, you can pass the parameter grid as a list of dictionaries specifying which combinations are allowed. The documentation specifies for such cases:

If a list of dicts is given, first a dict is sampled uniformly, and then a parameter is sampled using that dict as above.

So for example, let's say you defined the following as the parameter grid:

param_distribs = [
    {
        'kernel': ['rbf','poly'],
        'C': reciprocal(20, 200000),
        'gamma': expon(scale=1.0),
        'degree': expon(scale=1.0)
    },
    {
        'kernel': ['linear','sigmoid'],
        'C': reciprocal(20, 200000)
    }
]

This would avoid RandomizedSearchCV to set gamma and degree when it chooses the dictionary with linear kernel in an iteration. On the contrary, if it chooses the other dictionary in a particular iteration, it will set gamma and degree as well.