[英]Python: Using a list with TF-IDF
我有以下一段代码,当前将“令牌”中的所有单词与“ df”中的每个文档进行比较。 有什么办法可以将预定义的单词列表与文档(而不是“令牌”)进行比较。
from sklearn.feature_extraction.text import TfidfVectorizer
tfidf_vectorizer = TfidfVectorizer(norm=None)
list_contents =[]
for index, row in df.iterrows():
list_contents.append(' '.join(row.Tokens))
# list_contents = df.Content.values
tfidf_matrix = tfidf_vectorizer.fit_transform(list_contents)
df_tfidf = pd.DataFrame(tfidf_matrix.toarray(),columns= [tfidf_vectorizer.get_feature_names()])
df_tfidf.head(10)
任何帮助表示赞赏。 谢谢!
不确定我是否理解正确,但是如果您想让Vectorizer考虑固定的单词列表,则可以使用vocabulary
参数。
my_words = ["foo","bar","baz"]
# set the vocabulary parameter with your list of words
tfidf_vectorizer = TfidfVectorizer(
norm=None,
vocabulary=my_words)
list_contents =[]
for index, row in df.iterrows():
list_contents.append(' '.join(row.Tokens))
# this matrix will have only 3 columns because we have forced
# the vectorizer to use just the words foo bar and baz
# so it'll ignore all other words in the documents.
tfidf_matrix = tfidf_vectorizer.fit_transform(list_contents)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.