How does Support Vector Machine compare to Logistic Regression?

Question

Support Vector Machine (SVM) and logistic regression (LR) have been discussed widely in machine learning community, I know that both of them achieve pretty good performance. But, I am not sure how in general SVM compared to logistic regression? Why sometime SVM can perform better than LR? And sometime not? What factors determine these.

Answer 1

Why sometime SVM can perform better than LR? And sometime not?

You could pose this question for any two statistical methods x and y. There will always exist certain cases where one performs better than the other. This behaviour is often summarized by the words "there is no free lunch".

Now, your particular question on support vector machines and logistic regression is very broad, such that I can name only a few rough features. Logistic regression, ie a linear model to which a sigmoid is applied, is often used in practice because the fitted parameters are well interpretable. Further, as a (generalized) linear model it often performs well also for large feature dimensions. The SVM, on the other hand, often lead to smaller training and generalization errors, which due to the usage of a relatively small number of support vectors ("sparsity") is well applicable to data sets with large numbers of samples.

In practice, you should always compare both if possible. Further, I would suggest to prefer logistic regression if interpretability is required, and a SVM if a well prediction capability is desired.