How find the most decisive sentences or words in a document via Doc2Vec?

Question

I've trained a Doc2Vec model in order to do a simple binary classification task, but I would also love to see which words or sentences weigh more in terms of contributing to the meaning of a given text. So far I had no luck finding anything relevant or helpful. Any ideas how could I implement this feature? Should I switch from Doc2Vec to more conventional methods like tf-idf?

Answer 1

You are asking about model interpretability. Some ways I have seen this explored:

• Depending on your classifier, the parameters of the model may tell you what it is looking at. For example, in attention-based models, what the model attends to is telling.

• Tools like Lime and Anchor are useful for any black box model, and will probably work in this case. The documentation for both shows how to use it with text data.