测试性能分类器Matlab

Question

Hello I have trained a classifier in matlab and I would like to test its accuracy. I have found a lot of functions to do that and I am puzzling on what to use...

at the moment:

% train my classifier
svmStruct = fitcsvm(Xtrain,Ytrain,'KernelFunction','linear','Standardize',true);

% predict the output of an unknown input <- this part takes a lot of time
IDX_svm = zeros(size(Xtest,1),1);
for j = 1 : size(Xtest,1)    
    IDX_svm(j) = predict(svmStruct,Xtest(j,:));    
end

%compute performaces
TABLE = confusionmat(Ytest,IDX_svm);
sum_diag = 0;
for j = 1 : size(TABLE,2)
    sum_diag = sum_diag + TABLE(j,j);
end
error_rate = 1-(sum_diag/sum(sum(TABLE,2)));

Answer 1

The accuracy is simply defined as the ratio between the correctly predicted labels and the total number of labels in the testing/validation set. So instead of using the confusion matrix, if you have the Testing Labels vector ( Ytest I suppose) and the Predicted Labels vector ( IDX_svm I suppose), you can simply run

Accuracy=sum(IDX_svm==Ytest)/length(Ytest)

Accuracy will be in range [0;1], which you can scale in percentage by simply multiplying it by 100.

The Error Rate, of course, is defined according to one of the following:

• If you have previously evaluated the accuracy
  • 1-Accuracy if Accuracy is in range [0;1]
  • 100-Accuracy if Accuracy is given in percentage
• If you want to evaluate just the error rate
  • the ratio between the in correctly predicted labels and the total number of labels in the testing/validation set: ErrorRate=sum(IDX_svm~=Ytest)/length(Ytest) . Such value will be in range [0;1]
  • multiplying the above result by 100 will give the error rate in percentage

These are standard definitions and they work for every classifier, not just the SVM.

Also, I recommend to avoid the predict() function in a loop. In this case you call predict() several times and each call classifies a single point. This slows down your code. As you might now, predict() can take the whole testing matrix as input and returns the entire label vector so you don't have to preallocate and write every single element of such vector inside a for-loop. And you call predict() only once. You might want to try something like

IDX_svm=predict(svmStruct,Xtest);