Feature importance in sklearn using adaboost

Question

I am sing python library sklearn. I am using adaboost classifier and want to identify which features are most important in classification. Following is my code:

ada =    AdaBoostClassifier(n_estimators=100)
selector = RFECV(ada, step=1, cv=5) 
selector = selector.fit(np.asarray(total_data), np.asarray(target))
selector.support_
print "featue ranking", selector.ranking_

I am getting following error:

 selector = selector.fit(np.asarray(total_data), np.asarray(target))
  File "C:\Python27\lib\site-packages\sklearn\feature_selection\rfe.py", line 336, in fit
    ranking_ = rfe.fit(X_train, y_train).ranking_
  File "C:\Python27\lib\site-packages\sklearn\feature_selection\rfe.py", line 148, in fit
    if estimator.coef_.ndim > 1:
AttributeError: 'AdaBoostClassifier' object has no attribute 'coef_'

Does anyone have any idea about it, and how to correct it.

Thanks!!

Answer 1

Straight from the docstring of RFECV :

Parameters
----------
estimator : object
    A supervised learning estimator with a `fit` method that updates a
    `coef_` attribute that holds the fitted parameters. Important features
    must correspond to high absolute values in the `coef_` array.

    For instance, this is the case for most supervised learning
    algorithms such as Support Vector Classifiers and Generalized
    Linear Models from the `svm` and `linear_model` modules.

In other words, RFE is currently only implemented for linear models. You could make it work for other models by changing it to use feature_importances_ instead of coef_ and submit a patch.