keras 的特征重要性

Question

i have the following question. I created a neural network with Keras. That is a Part of my dataFrame. My DataFrame looks like:

    Id          MainCl   Class     Other Options...
    1016178069  0        30        1              
    1016178012  0        25        0              

My result vector gives me the probability in which class the result can be classified. Example:

Prob_isClass_A    Prob_isClass_b
0.756686          0.243314
0.999843          0.000157006
0.999818          0.000182159

Now to the main question: Is there a way to get the highest probability of a feature that is responsible for the result? Like:

Prob_isClass_A    Prob_isClass_b      Most_Probability_Reason    Probability
0.756686          0.243314            Main_CL_Feature            0.75
0.999843          0.000157006         Class_Feature              0.8
0.999818          0.000182159         Other_Option_Feature       0.7

Answer 1

I answered a related question at Feature Importance Chart in neural network using Keras in Python . The only difference I can see here is that rather looking for an explanation of the feature importance for the ensemble metric, you want feature importance per individual prediction. In this case, the same package, ELI5 , has functions which can explain individual predictions as well in eli5.sklearn.explain_prediction . I haven't yet had the need to use this functionality, but the documentation there seems straightforward.

Answer 2

If i understand your question, you want something like first of feature importance of every sample separately ...

Feature importance isn't supported by keras. You have to look in tree models for something like that. Furthermore you need something more that and as far as i can tell, has not yet implemented and probably wont be.