在R中实现核岭回归

Question

I want to implement kernel ridge regression in R. My problem is that I can't figure out how to generate the kernel values and I do not know how to use them for the ridge regression. I want to use the following kernel function:

kernel.eval <- function(x1,x2,ker) { k=0 if (kertype == 'RBF') {
    # RBF kernel
    k=exp(-sum((x1-x2)*(x1-x2)/(2*kerparam^2))) } else { # polynomial kernel k=(1+sum(x1*x2))^ker$param } return(k) }

Furthermore, I know that the formula for ridge regression is:

myridge.fit <- function(X,y,lambda) { w= solve((t(X) %% X) +(lambdadiag(dim(X)[2])), (t(X) %*% y)) return(w) }

Example training data:

           [,1]        [,2] 
[1,] -1.3981847 -1.3358413 
[2,] 0.2698321   1.0661275 
[3,] 0.3429286   0.8805642 
[4,] 0.5210577   1.1228635 
[5,] 1.5755659   0.2230754 
[6,] -1.2167197 -0.6700215

Example testing data: (I do not know if I need these at this moment)

      [,1]   [,2] 
[1,] -2.05 -2.050 
[2,] -2.05 -2.009 
[3,] -2.05 -1.968 
[4,] -2.05 -1.927 
[5,] -2.05 -1.886 
[6,] -2.05 -1.845

Is anyone able to help me with the first step(s). I have to do Ridge Regression for a RBF kernel as well as a Polynomial kernel.

Answer 1

Following is the code for polynomial kernel with degree 2, hope that helps!

poly.kernel <- function(v1, v2=v1, p=2) {   
    ((as.matrix(v1) %*% t(v2))+1)^p
}   

KernelRidgeReg <- function(TrainObjects,TrainLabels,TestObjects,lambda){

  X <- TrainObjects
  y <- TrainLabels                      
  kernel <- poly.kernel(X)

  design.mat <- cbind(1, kernel)

  I <- rbind(0, cbind(0, kernel))

  M <- crossprod(design.mat) + lambda*I
  #crossprod is just x times  traspose of x, just looks neater in my openion

  M.inv <- solve(M)
  #inverse of M

  k <- as.matrix(diag(poly.kernel(cbind(TrainObjects,TrainLabels))))
  #Removing diag still gives the same MSE, but will output a vector of prediction.

  Labels <- rbind(0,as.matrix(TrainLabels))

  y.hat <- t(Labels) %*% M.inv %*% rbind(0,k)

  y.true <- Y.test

  MSE <-mean((y.hat - y.true)^2) 

  return(list(MSE=MSE,y.hat=y.hat))

}

Solve built-in R function sometimes return singular matrix. You may want to write your own function to avoid that.