How to get the scores of each feature from sklearn.feature_selection.SelectKBest?

Question

I am trying to get the scores of all the features of my data set.

file_data = numpy.genfromtxt(input_file)
y = file_data[:,-1]
X = file_data[:,0:-1]

x_new = SelectKBest(chi2, k='all').fit_transform(X,y)

Before the first row of X had the "Feature names" in string format but I was getting "Input contains NaN, infinity or a value too large for dtype('float64')" error. So, now X contains only the data and y contains the target values(1,-1).

How can I get the score of each feature from SelectKBest(trying to use Uni-variate feature selection)?

thanks

Answer 1

Solution

You just have to do something like this.

file_data = numpy.genfromtxt(input_file)
y = file_data[:,-1]
X = file_data[:,0:-1]

selector = SelectKBest(chi2, k='all').fit(X,y)
x_new = selector.transform(X) # not needed to get the score
scores = selector.scores_

Your problem

When you use directly .fit_transform(features, target) , the selector is not stored and you are returning the selected features . However, the scores is an attribute of the selector . In order to get it, you have to use .fit(features, target) . Once you have your selector fitted, you can get the selected features by calling selector.transform(features) , as you can see in the code avobe.

As I commented in the code, you don't need to have transformed the features to get the score. Just with fitting them is enough.

Links

• Documentation about SelectKBest in sklearn
• Example in the docs