How Support Vector Regression works?

Question

I'm trying to understand SVR model.
To do it I looked at SVM and it's pretty clear for me. But there is no much explications about SVR. The first question is why it's called Support Vector Regression or how we use vectors to predict numerical values?
Also I don't understand some parameters such as epsilon and gamma . How they influence predicted result?

Answer 1

A SVM learns a so called decision function from your features, such that features from you positive class produce positive real numbers, and features from the negative class produce negative numbers (at least most of the time, depending on your data).

For two features you can visualize this in a 2D plane. The function assigns a real value to each point in the plane, this value can be depicted as color. This plot shows the values as different blue colors.

The feature values resulting in zero form the so called decision boundary .

This function itself has two kind of parameters:

• kernel dependend parameters. In your case for the radial basis functions, these parameters are epsilon and gamma , which you set before learning.
• And the so called support-vectors which are determined during learning. support-vectors are just parameters of your decision function .

Learning is nothing than determining good support-vectors (parameters !).

In this 2d example video the colors don't show the actual function value, but only the sign. You can see how gamma influences the smoothness of the decision function.

To answer you question:

SVR builds such a function but with a different goal. The function does not try to assign positive outcomes to your postive examples, and negative outcomes to the negative examples.

Instead the function is built to approximate the given numeric outcomes.