如何使用线性支持向量机（SVM）分类器确定最重要/信息功能

Question

I am new to python and working on a text classification problem. I am interested in the visualization of the most important features of each class through a linear SVM classifier model. I want to determine which features are contributing towards the classification decision as Class-1 or Class-2 by classification model. This is my code.

df = pd.read_csv('projectdatacor.csv')
df = df[pd.notnull(df['types'])]
my_types = ['Requirement','Non-Requirement']

#converting to lower case
df['description'] = df.description.map(lambda x: x.lower()) 

#Removing the punctuation
df['description'] = df.description.str.replace('[^\w\s]', '')  


#splitting the word into tokens
df['description'] = df['description'].apply(nltk.tokenize.word_tokenize) 


## This converts the list of words into space-separated strings
df['description'] = df['description'].apply(lambda x: ' '.join(x))
count_vect = CountVectorizer()  
counts = count_vect.fit_transform(df['description']) 


#tf-idf
transformer = TfidfTransformer().fit(counts)
counts = transformer.transform(counts)  

#splitting the data and training the model
#naives-bayes
X_train, X_test, y_train, y_test = train_test_split(counts, df['types'], test_size=0.3, random_state=39)

#svc classification
from sklearn import svm
svclassifier = svm.SVC(gamma=0.001, C=100., kernel = 'linear')

svclassifier.fit(X_train, y_train)
y_pred = svclassifier.predict(X_test) 

#evalutaing the model
print(classification_report(y_test,y_pred))
print(confusion_matrix(y_test,y_pred))  
print('accuracy %s' % accuracy_score(y_pred, y_test))
print(classification_report(y_test, y_pred,target_names=my_types))

I have read all related questions available on this platform but I found the following useful code which I added in my code.

import numpy as np 
def show_most_informative_features(vectorizer, clf, n=20): 
    feature_names = vectorizer.get_feature_names() 
    coefs_with_fns = sorted(zip(clf.coef_[0], feature_names)) 
    top = zip(coefs_with_fns[:n], coefs_with_fns[:-(n + 1):-1]) 
    for (coef_1, fn_1), (coef_2, fn_2) in top: 
        print ("\t%.4f\t%-15s\t\t%.4f\t%-15s")  % (coef_1, fn_1, coef_2, fn_2) 
show_most_informative_features(count_vect, svclassifier, 20)

This code works for naive Bayes and logistic regression and it gives the most important features but for SVM it gives me the error.

I am getting this error.

  File "C:\Users\fhassan\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 704, in runfile
    execfile(filename, namespace)

  File "C:\Users\fhassan\anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 108, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "U:/FAHAD UL HASSAN/Python Code/happycsv.py", line 209, in <module>
    show_most_informative_features(count_vect, svclassifier, 20)

  File "U:/FAHAD UL HASSAN/Python Code/happycsv.py", line 208, in show_most_informative_features
    print ("\t%.4f\t%-15s\t\t%.4f\t%-15s" % (coef_1, fn_1, coef_2, fn_2))

TypeError: must be real number, not csr_matrix

Any help shall be highly appreciated.

Answer 1

Maybe this will help you:

from sklearn import svm
import pandas as pd
import numpy as np 

from sklearn.feature_extraction.text import CountVectorizer
corpus = [
    'This is the first document.',
    'This document is the second document.',
    'And this is the third one.',
    'Is this the first document?']

vectorizer = CountVectorizer()
X = vectorizer.fit_transform(corpus)

x=X.toarray()
y=[0,0,0,1]

model=svm.SVC(kernel='linear')

a=model.fit(x,y)
model.score(x,y)

feature_names = vectorizer.get_feature_names() 
coefs_with_fns = sorted(zip(model.coef_[0], feature_names)) 
df=pd.DataFrame(coefs_with_fns)
df.columns='coefficient','word'
df.sort_values(by='coefficient')

You will get: