ILNumerics: ILMath.ridge_regression

Question

is there anyone know how to use ridge_regression in ILMath function? I try to read the documents and search in several website, but I can't find the example.

Here is the method:

public static ILMath..::..ILRidgeRegressionResult<double> ridge_regression(
     ILInArray<double> X,
     ILInArray<double> Y,
     ILBaseArray Degree,
     ILBaseArray Regularization
)

click here to see the details of the function

I had a bit confuse with the "Regularization".

Answer 1

ILNumerics.ILMath.ridge_regression

Basically, ridge_regression learns a polynomial model from some test data. It works in a two step process:

1) In the learning phase you create a model. The model is represented by an instance of the ILRidgeRegressionResult class which is returned from ridge_regression:

using (var result = ridge_regression(Data,Labels,4,0.01)) {
   // the model represented by 'result' is used within here
   // to apply it to some unseen data... See step 2) below. 
   L.a = result.Apply(X + 0.6); 
}

Here, X is some data set and Y is the set of 'labels' which corresponds to those X data. In this example, X is a linear vector and Y is the result of the sin() function on that vector. So the ridge_regression result represents a model which produces similar results as the sin() function - in certain limits. In real applications, X may be of any dimensionality.

2) Apply the model: the regression result is than used to estimate values corresponding to new, unseen data. We apply the model to data, which have the same number of dimensions as the original data. But some of the data points lay within the range, some outside of the range we used to learn the data from. The apply() function of the regression result object therefore allows us to interpolate as well as extrapolate data.

The complete example:

private class Computation : ILMath {
    public static void Fit(ILPanel panel) {
        using (ILScope.Enter()) {
            // just some data 
            ILArray<double> X = linspace(0, 30, 20) / pi / 4;
            // the underlying function. Here: sin()
            ILArray<double> Y = sin(X);

            // learn a model of 4th order, representing the sin() function
            using (var result = ridge_regression(X, Y, 4, 0.002)) {
                // the model represented by 'result' is used within here
                // to apply it to some unseen data... See step 2) below. 
                ILArray<double> L = result.Apply(X + 0.6); 

                // plot the stuff: create a plotcube + 2 line XY-plots
                ILArray<double> plotData = X.C; plotData["1;:"] = Y; 
                ILArray<double> plotDataL = X + 0.6; plotDataL["1;:"] = L; 
                panel.Scene.Add(new ILPlotCube() {
                    new ILLinePlot(tosingle(plotData), lineColor: Color.Black, markerStyle:MarkerStyle.Dot), 
                    new ILLinePlot(tosingle(plotDataL), lineColor: Color.Red, markerStyle:MarkerStyle.Circle), 
                    new ILLegend("Original", "Ridge Regression")
                }); 
            }
        }
    }
}

This produces the following result:

Some Notes

1) Use ridge_regression in a 'using' block (C#). This ensures that the data of the model which can be quite large are disposed off correctly.

2) The Regularization becomes more important, once you try to learn a model from data which may introduce some stability problems. You need to experiment with the regularization term and take the actual data into account.

3) In this example, you see the interpolation result fitting very nicely the original function. However, the underlying model is based on a polynomial. As common for (all/polynomial) models, the estimated values may reflect the underlying model less and less, the far you get from the original range of values used in the learning phase.