Compute the best cost function - SVM

Question

I have to calculate the Cost Function for the following classification problem, using the SVM:

Training data:

X1   X2   Y  
1.3  0.2  0
1.5  0.4  0
4.7  1.4  1
4.5  1.5  1

A. 1.6*x1 + 4*x2 - 5.6 = 0
B. 2.4*x1 + 4 * x2 - 7.2 = 0
C. 0.96*x1 + 4 * x2 - 4.8 = 0

How to calculate the Cost function for each decision boundaries above, to find the best?

Answer 1

To calculate the "cost" of each given decision boundary, you have to calculate the predicted label for each data point. For example, for the decision boundary A and the first data point, you have to substitute x1 = 1.3 and x2 = 0.2 and then the predicted label is 1.6 * 1.3 + 4 * 0.2 - 5.6 = -2.72 . Since you are dealing with binary label, normally the model (you're not training a model and the decision boundary is already given) should say something like "if (1.6*x1 + 4*x2 - 5.6) >= 0, then its label is 1" (or vice versa). Double check the given model. For example, let's assume the predicted label for the decision boundary A and the first data point is 0 (since the computed value is -2.72 < 0) then this is a true negative (since the predicted label and the given label are both 0 ).

There are 4 cases in total: 1) If predicted label is 1 and the given label is 1, it's called "true positive". 2) If predicted label is 0 and the given label is 0, it's called "true negative". 3) If predicted label is 1 and the given label is 0, it's called "false positive". 4) If predicted label is 0 and the given label is 1, it's called "false negative".

Then, usually, the cost function is a function in term of the number of "true positive", "true negative", "false positive", "false negative". Then, after you compute the predicted labels for all four given data points, you can calculate these numbers and hence the cost of the model.

PS In 99.9% of chance, cost function will be in term of "false positive", "false negative" only, but it could sometimes also depend on "true positive", "true negative" in some very rare cases.