使用scikit-learn进行文档分类：获取对分类影响更大的单词（令牌）的最有效方法

Question

I have built a document binomial classifier using a tf-idf representation of a training set of documents and applying Logistic Regression to it:

lr_tfidf = Pipeline([('vect', tfidf),('clf', LogisticRegression(random_state=0))])

lr_tfidf.fit(X_train, y_train)

I have saved the model in pickle format and used it to classify new documents, getting as a result the probability the document is in class A and the probability the model is in class B.

text_model = pickle.load(open('text_model.pkl', 'rb'))
results = text_model.predict_proba(new_document)

Which is the best way to also get the words (or, in general, the tokens) that impacted more in the classification? I would expect to get:

• The N tokens contained in the document which had the higher coefficient as a feature in the Logistic Regression model
• The N tokens contained in the document which had the lower coefficient as a feature in the Logistic Regression model

I am using sklearn v 0.19

Answer 1

There is a solution on GitHub to print the most important features obtained from a classifier within a pipeline:

https://gist.github.com/bbengfort/044682e76def583a12e6c09209c664a1

You want to use show_most_informative_features function in their script. I used it and it works fine.

Here is a copy-paste of the Github poster's code:

def show_most_informative_features(model, text=None, n=20):

"""

Accepts a Pipeline with a classifer and a TfidfVectorizer and computes

the n most informative features of the model. If text is given, then will

compute the most informative features for classifying that text.



Note that this function will only work on linear models with coefs_

"""

# Extract the vectorizer and the classifier from the pipeline

vectorizer = model.named_steps['vectorizer']

classifier = model.named_steps['classifier']



# Check to make sure that we can perform this computation

if not hasattr(classifier, 'coef_'):

    raise TypeError(

        "Cannot compute most informative features on {} model.".format(

            classifier.__class__.__name__

        )

    )



if text is not None:

    # Compute the coefficients for the text

    tvec = model.transform([text]).toarray()

else:

    # Otherwise simply use the coefficients

    tvec = classifier.coef_



# Zip the feature names with the coefs and sort

coefs = sorted(

    zip(tvec[0], vectorizer.get_feature_names()),

    key=itemgetter(0), reverse=True

)



topn  = zip(coefs[:n], coefs[:-(n+1):-1])



# Create the output string to return

output = []



# If text, add the predicted value to the output.

if text is not None:

    output.append("\"{}\"".format(text))

    output.append("Classified as: {}".format(model.predict([text])))

    output.append("")



# Create two columns with most negative and most positive features.

for (cp, fnp), (cn, fnn) in topn:

    output.append(

        "{:0.4f}{: >15}    {:0.4f}{: >15}".format(cp, fnp, cn, fnn)

    )



return "\n".join(output)

Answer 2

Here is a modified version of the show_most_informative_features function that works with any classifier:

def show_most_informative_features(model, vectorizer=None, text=None, n=20):
# Extract the vectorizer and the classifier from the pipeline
if vectorizer is None:
    vectorizer = model.named_steps['vectorizer']
else:
    vectorizer.fit_transform([text])

classifier = model.named_steps['classifier']
feat_names = vectorizer.get_feature_names()

# Check to make sure that we can perform this computation
if not hasattr(classifier, 'coef_'):
    raise TypeError(
        "Cannot compute most informative features on {}.".format(
            classifier.__class__.__name__
        )
    )    

# Otherwise simply use the coefficients
tvec = classifier.coef_

# Zip the feature names with the coefs and sort   
coefs = sorted(
    zip(tvec[0], feat_names),
    key=operator.itemgetter(0), reverse=True
)

# Get the top n and bottom n coef, name pairs
topn  = zip(coefs[:n], coefs[:-(n+1):-1])

# Create the output string to return
output = []

# If text, add the predicted value to the output.
if text is not None:
    output.append("\"{}\"".format(text))
    output.append(
        "Classified as: {}".format(model.predict([text]))
    )
    output.append("")

# Create two columns with most negative and most positive features.
for (cp, fnp), (cn, fnn) in topn:
    output.append(
        "{:0.4f}{: >15}    {:0.4f}{: >15}".format(
            cp, fnp, cn, fnn
        )
    )

return "\n".join(output)

Then you can call the function like this:

vectorizer = TfidfVectorizer()
show_most_informative_features(model,vectorizer, "your text")

Answer 3

From my understanding, you just want to look at the parameters and sort the according to the coefficient value. With .get_params() function, you can get the coefficients. You can argsort it and select the top N, bot N.