![](/img/trans.png)
[英]Unable to encode user input values that is given to the loaded Naive Bayes model?
[英]Unable to train model in Naive Bayes
我正在尝试使用 NLTK 将电子邮件分类为垃圾邮件/火腿
以下是遵循的步骤:
试图提取所有令牌
获取所有特征
从所有唯一词的语料库中提取特征并映射真/假
from nltk.classify.util import apply_features
from nltk import NaiveBayesClassifier
import pandas as pd
import collections
from sklearn.model_selection import train_test_split
from collections import Counter
data = pd.read_csv('https://raw.githubusercontent.com/venkat1017/Data/master/emails.csv')
"""fetch array of tuples where each tuple is defined by (tokenized_text, label)
"""
processed_tokens=data['text'].apply(lambda x:([x for x in x.split() if x.isalpha()]))
processed_tokens=processed_tokens.apply(lambda x:([x for x in x if len(x)>3]))
processed_tokens = [(i,j) for i,j in zip(processed_tokens,data['spam'])]
"""
dictword return a Set of unique words in complete corpus.
"""
list = zip(*processed_tokens)
dictionary = Counter(word for i, j in processed_tokens for word in i)
dictword = [word for word, count in dictionary.items() if count == 1]
"""maps each input text into feature vector"""
y_dict = ( [ (word, True) for word in dictword] )
feature_vec=dict(y_dict)
"""Training"""
training_set, testing_set = train_test_split(y_dict, train_size=0.7)
classifier = NaiveBayesClassifier.train(training_set)
~\AppData\Local\Continuum\anaconda3\lib\site-packages\nltk\classify\naivebayes.py in train(cls, labeled_featuresets, estimator)
197 for featureset, label in labeled_featuresets:
198 label_freqdist[label] += 1
--> 199 for fname, fval in featureset.items():
200 # Increment freq(fval|label, fname)
201 feature_freqdist[label, fname][fval] += 1
AttributeError: 'str' object has no attribute 'items'
我在尝试训练唯一词的语料库时遇到以下错误
首先,我希望您知道y_dict
只是一个字典,它将在语料库中仅出现一次的单词(字符串)映射为值True
键。 您将它作为训练集传递给分类器,而您应该传递一个tuple
(每个文本行的特征字典)和(相应的标签)。 虽然您的分类器应该接收[({'feat1': 'value1', ... }, label_value), ...]
作为输入,但您正在传递[ ('word1', True), ... ]
。 string
类型没有items
属性,只有dict
有。 因此错误。
其次,你的数据建模是错误的。 您的训练集应包含从映射到data['spam']
值的data['text']
派生的特征字典(因为这是您的标签)。 请在此处查看第 1.3 节中如何使用 nltk 的分类器执行文档分类。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.