TF-IDF Matrix In Python

Question

My code to calculate TF-IDF for a corpus goes like this:

from sklearn.feature_extraction.text import CountVectorizer
from sklearn.feature_extraction.text import TfidfTransformer

train_set = "i have a ball", "he is good", "she played well" 
vectorizer = TfidfVectorizer(min_df=1)

train_array = vectorizer.fit_transform(train_set).toarray()
print(vectorizer.get_feature_names())
print(train_array)

The output I receive is:

['ball', 'good', 'have', 'he', 'is', 'played', 'she', 'well']

[[0.70710678, 0., 0.70710678, 0., 0., 0., 0., 0.],
 [0., 0.57735027, 0., 0.57735027, 0.57735027, 0., 0., 0.],
 [0., 0., 0., 0., 0., 0.57735027, 0.57735027, 0.57735027]]

The question is how can I calculate TF-IDF of the sentence: "she is good" ? The corpus is the train_set in the above code.

Answer 1

You simply apply your TF-IDF vectorizer on new data with .transform method:

In [16]: test = ["she is good"]

In [17]: test_array = vectorizer.transform(test)

In [18]: test_array.A
Out[18]: array([[0., 0.57735027, 0., 0., 0.57735027, 0., 0.57735027, 0.]])

In [19]: vectorizer.get_feature_names()
Out[19]: ['ball', 'good', 'have', 'he', 'is', 'played', 'she', 'well']