sklearn.ensemble中的python特征选择特征重要性方法在多次运行中给出不一致的结果

Question

I am trying to do feature selection using python feature imporance from sklearn.ensemble. The problem is every time I run the code (below), the results varies. I mean it gives me different columns as the largest feature importance values. Isn't it strange? or am I doing something wrong (?)

I have too many features (about 500 ... & 50k records). I would like to get the more important features to improve the classification. But the results of feature importance doesn't seem consistant.

#Feature importance 
from sklearn.ensemble import ExtraTreesClassifier
import matplotlib.pyplot as plt

#X independednt cols and y the target col
model = ExtraTreesClassifier()
model.fit(X,y)

# print(model.feature_importances_)

feat_importances = pd.Series(model.feature_importances_, index=X.columns)
feat_importances.nlargest(20).plot(kind ="barh")

Answer 1

Randomness enters the fitting, so you should not expect to end up with the exact same results. To get reproducible results, you can provide the seed parameter to your estimator.

If for different seeds you end up with hugely different variable importances, this means that none of the features seems to dominate the predictive content of your data, as far as trees can capture it. So variable importances should be considered with a grain of salt.