ScikitLearn GridSearchCV 和管道使用不同的方法

Question

I am trying to evaluate these Machine Learning methods to the same data using GridSearchCV and pipeline, when I vary the parameters in the same method it works, but when I put multiple Methods it gives an error

pipe_steps = [
    ('scaler', StandardScaler()), 
    ('logistic', LogisticRegression()),
    ('SVM',SVC()),
    ('KNN',KNeighborsClassifier())]
check_params={
    'logistic__C':[1,1e5],
    'SVM__C':[1,1e5],
    'KNN__n_neighbors':[3,5],
    'KNN__metric':['euclidean','manhattan']
    }
pipeline = Pipeline(pipe_steps)
GridS = GridSearchCV(pipeline, param_grid=check_params)
GridS.fit(X, y)
print('Score %3.2f' %GridS.score(X, y))
print('Best Fit')
print(GridS.best_params_)

gives the error message on pipeline line below

TypeError                                 Traceback (most recent call last)
<ipython-input-139-75960299bc1c> in <module>
     13     }
     14 
---> 15 pipeline = Pipeline(pipe_steps)
     16 
     17 BCX_Grid = GridSearchCV(pipeline, param_grid=check_params)

C:\ProgramData\Anaconda3\lib\site-packages\sklearn\pipeline.py in __init__(self, steps, memory, verbose)
    133     def __init__(self, steps, memory=None, verbose=False):
    134         self.steps = steps
--> 135         self._validate_steps()
    136         self.memory = memory
    137         self.verbose = verbose

C:\ProgramData\Anaconda3\lib\site-packages\sklearn\pipeline.py in _validate_steps(self)
    183                                 "transformers and implement fit and transform "
    184                                 "or be the string 'passthrough' "
--> 185                                 "'%s' (type %s) doesn't" % (t, type(t)))
    186 
    187         # We allow last estimator to be None as an identity transformation

TypeError: All intermediate steps should be transformers and implement fit and transform or be the string 'passthrough' 'LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
                   intercept_scaling=1, l1_ratio=None, max_iter=100,
                   multi_class='warn', n_jobs=None, penalty='l2',
                   random_state=None, solver='warn', tol=0.0001, verbose=0,
                   warm_start=False)' (type <class 'sklearn.linear_model.logistic.LogisticRegression'>) doesn't

Thanks

Answer 1

Your problem is not the hyperparameters as they are defined correctly. The problem is that all intermediate step should be transformers , as the error indicates. in your pipeline SVM is not a transformer.

see this post

Answer 2

You need to split the pipeline into multiple pipelines, for that I have a solution that requires a list of grid params that determines each step of the pipeline.

pipeline = Pipeline([
    ('transformer', StandardScaler(),),
    ('model', 'passthrough',),
])

params = [
    {
        'model': (LogisticRegression(),),
        'model__C': (1, 1e5,),
    },
    {
        'model': (SVC(),),
        'model__C': (1, 1e5,),
    },
    {
        'model': (KNeighborsClassifier(),),
        'model__n_neighbors': (3, 5,),
        'model__metric': ('euclidean', 'manhattan',),
    }
]

grid_Search = GridSearchCV(pipeline, params)

With this strategy you can define the steps of the pipeline dynamically.