Scikit-learn using GridSearchCV on DecisionTreeClassifier

Question

I tried to use GridSearchCV on DecisionTreeClassifier, but get the following error: TypeError: unbound method get_params() must be called with DecisionTreeClassifier instance as first argument (got nothing instead)

here's my code:

from sklearn.tree import DecisionTreeClassifier, export_graphviz
from sklearn.model_selection import GridSearchCV
from sklearn.cross_validation import  cross_val_score

X, Y = createDataSet(filename)
tree_para = {'criterion':['gini','entropy'],'max_depth':[4,5,6,7,8,9,10,11,12,15,20,30,40,50,70,90,120,150]}
clf = GridSearchCV(DecisionTreeClassifier, tree_para, cv=5)
clf.fit(X, Y)

Answer 1

In your call to GridSearchCV method, the first argument should be an instantiated object of the DecisionTreeClassifier instead of the name of the class. It should be

clf = GridSearchCV(DecisionTreeClassifier(), tree_para, cv=5)

Check out the example here for more details.

Hope that helps!

Answer 2

Another aspect regarding the parameters is that grid search can be run with different combination of parameters. The parameters mentioned below would check for different combinations of criterion with max_depth

tree_param = {'criterion':['gini','entropy'],'max_depth':[4,5,6,7,8,9,10,11,12,15,20,30,40,50,70,90,120,150]}

If needed, the grid search can be run over multiple set of parameter candidates:

For example:

tree_param = [{'criterion': ['entropy', 'gini'], 'max_depth': max_depth_range},
              {'min_samples_leaf': min_samples_leaf_range}]

In this case, grid search would be run over two sets of parameters, first with every combination of criterion and max_depth and second, only for all provided values of min_samples_leaf

Answer 3

需要在分类器前加一个()：

clf = GridSearchCV(DecisionTreeClassifier(), tree_para, cv=5)

Answer 4

Here is the code for decision tree Grid Search

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV

def dtree_grid_search(X,y,nfolds):
    #create a dictionary of all values we want to test
    param_grid = { 'criterion':['gini','entropy'],'max_depth': np.arange(3, 15)}
    # decision tree model
    dtree_model=DecisionTreeClassifier()
    #use gridsearch to test all values
    dtree_gscv = GridSearchCV(dtree_model, param_grid, cv=nfolds)
    #fit model to data
    dtree_gscv.fit(X, y)
    return dtree_gscv.best_params_

Answer 5

If the problem is still there try to replace :

from sklearn.grid_search import GridSearchCV

with

from sklearn.model_selection import GridSearchCV

It sounds stupid but I had similar problems and I managed to solve them using this tip.