为机器学习选择最佳特征

Question

Is there any way to extract best features from the data. Right now, I am using 'KBest' from sklearn. In this, I have to specify number of K best features that needs to be selected. Is there any way in which I don't have to specify the number of features to be extracted? Rather we extract all the useful features?

from sklearn.feature_selection import SelectKBest
test = SelectKBest(score_func=chi2, k=4)

Answer 1

You can use "all" instead of a number

test = SelectKBest(score_func=chi2, k="all")

From docs

k : int or “all”, optional, default=10

Number of top features to select. The “all” option bypasses selection, for use in a parameter search.

Answer 2

Many ways to select features. In wiki , you can find them.And I think the best feature selection method is that you have a deep understanding of these features.But usually we have a hard time understanding them.

Maybe you can use 5-Kfold cross-validation to make a feature importance ranking, and them select important feature from it.

And you also can use Embedded method to select it, like this:

from sklearn.feature_selection import SelectFromModel
from sklearn.ensemble import GradientBoostingClassifier

#Feature selection of GBDT as base model
SelectFromModel(GradientBoostingClassifier()).fit_transform(iris.data, iris.target)

It's worth noting that you cannot delete a feature that seems to be useless alone,because it may be related to other features.So feature selection is a greedy search process, which is often time consuming.