libsvm的准确性不正确

Question

I am doing libsvm as shown there:

numLabels = max(trainlabels);
model = cell(numLabels,1);
for k=1:numLabels
    model{k} = svmtrain(double(trainlabels==k), train_data, '-t 0 -b 1');
end

numTest = size(test_data,1);
prob = zeros(numTest,numLabels);
for k=1:numLabels
    [predicted_label, accuracy, prob_estimates] = svmpredict(double(testlabels==k), test_data, model{k}, '-b 1');
    fprintf('sum(predicted_label==1) = %i \n', sum(predicted_label==1));
    fprintf('sum(testlabels==k) = %i \n', sum(testlabels==k));
    cc = model{k}.Label==1;
    prob(:,k) = prob_estimates(:,cc);    %# probability of class==k
end

%# predict the class with the highest probability
[~,pred] = max(prob,[],2);
acc = sum(pred == testlabels) ./ numel(testlabels);    %# accuracy
fprintf('Final accuracy = %f %\n', acc*100);

I got accuracy always above 80% (~83%), and sum of predicted label of each iteration is almost zero, and final accuracy is 11%. As shown below:

Accuracy = 65% (39/60) (classification)
sum(predicted_label==1) = 17 
Accuracy = 83.3333% (50/60) (classification)
sum(predicted_label==1) = 0 
Accuracy = 83.3333% (50/60) (classification)
sum(predicted_label==1) = 0 
Accuracy = 63.3333% (38/60) (classification)
sum(predicted_label==1) = 16 
Accuracy = 83.3333% (50/60) (classification)
sum(predicted_label==1) = 0 
Accuracy = 83.3333% (50/60) (classification)
sum(predicted_label==1) = 0 
Final accuracy = 11.666667 >> 

How this happens ? What I am doing wrong here ???

Answer 1

In my opinion, your code is fine. The reason you get the low accuracy is that svm does not work well on this dataset.

As you can see, in several model, your svm just predicts everything to be negative. This means it is not that confident for predicting the positive label. Therefore, when you combine all the probability in the end, the accuracy is bad.

I suggest you to try another kernel with svm to see the result.