如何优化使用拉普拉斯平滑处理的文本分类器，以便在大型数据集上更快地执行

Question

我正在尝试使用来自twitter的数据集创建一个文本分类器，用于对两类分类的文本进行分类：积极或正常。

我的分类器正在以66％的准确度工作，但由于数据集非常庞大，因此速度非常慢。

任何建议我怎样才能让它更快？

数据集可以在这里看到：

https://dataturks.com/projects/abhishek.narayanan/Dataset%20for%20Detection%20of%20Cyber​​-Trolls

我的主要代码：

'''
The dataset basically contains vectors with two components.
The first component is basically the String to analyse.
The Second component is the integer representing the sort of text
       1 ----> Aggressive Text
       0 ----> Positive Text
'''
import Preprocessdata as Dataset
import random


def Dataset_by_class(Dataset):
    Dict = {}
    for Instance in Dataset:
        if Instance[-1] not in Dict:
            Dict[Instance[-1]] = []
        Dict[Instance[-1]].append(Instance[0])
    return Dict

def String_by_class(Dataset):
    Dict = {}
    for index,Instance in Dataset.items():
        Final = []
        for Value in Instance:
            Final.extend(Value.split())
        Dict[index] = Final
    return Dict

def PreProcess_TestSet(Dataset):
    Label = []
    Testing = []
    for Instance in Dataset:
        Testing.append(Instance[0])
        Label.append(Instance[1])
    return[Testing,Label]

def Laplace_Smoothening(Item,List,Alpha,Unique_Count,index):
    Prob = 1
    for String in List:
        X = Item.count(String)
        Prob = Prob * ((X+Alpha)/(len(Item) + Alpha *  Unique_Count[index]))
    return Prob


def Predict(Dataset,TestSet):
    # We will be using LAplace Smoothening using alpha as 1
    Unique_Count = {}
    for index,Item in Dataset.items():
        Unique_Count[index] = len(set(Item))
    Alpha = 1
    Predicted_Label = []
    Prob_by_class = {}
    for Test_Instance in TestSet:
        List = Test_Instance.split()
        for index,Item in Dataset.items():
            Probility_by_class = Laplace_Smoothening(Item,List,Alpha,Unique_Count,index)
            Prob_by_class[index] = Probility_by_class
        if(Prob_by_class[0] > Prob_by_class[1]):
            Predicted_Label.append(0)
        else:
            Predicted_Label.append(1)
    return Predicted_Label

def Calculate_Accuracy(Prediction,Labels):
    Count = 0
    for Predicted,Given in zip(Prediction,Labels):
        if Predicted == Given:
            Count += 1
    return (Count/len(Labels))*100


# This function automatically creates dataset from json and is imported

Data_Set = Dataset.PreProcess_Dataset('Dataset for Detection of Cyber-Trolls.json') 

SplitRatio = 0.70

#Splits the dataset according to split ratio

TrainSet,TestSet = Train_Test_Dataset(Data_Set,SplitRatio) 

DataSet_By_Class = Dataset_by_class(TrainSet)
String_By_Class = String_by_class(DataSet_By_Class)
TestSet,Labels = PreProcess_TestSet(TestSet)
Prediction = Predict(String_By_Class,TestSet)
Accuracy = Calculate_Accuracy(Prediction,Labels)
print("The Accuracy is {:.2f}".format(Accuracy))

结果很好，但速度非常慢。