Creating a Radial basis function kernel matrix in matlab

Question

I never used matlab, and I have this code about kernalized locality sensitive functions .

I think that the following code is trying to create the kernalized matrix of a RBF kernel function :

%demo script for KLSH
X = load('iris.mtx');
...
[n,d] = size(X);

%form RBF over the data:
nms = sum(X'.^2);
K = exp(-nms'*ones(1,n) -ones(n,1)*nms + 2*X*X');

You can find the whole code here and in particular this code in demo.m .

Now, I cannot find the correlation of how K (the kernel matrix) is computed and the kernel function formula:

Can you help me to figure out how K is created (and explain me the code above) please?

Answer 1

The whole trick is based on the fact that you want to compute matrix K_ij = K(x_i, x_j) = f(||x_i - x_j||^2) in an efficient manner. Matrix computations are based on dot products, thus multiplications, not on norm of a difference. If you do not want to use loops (and in languages like matlab or R you do not want to) you have to figure out how to express this ||x_i - x_j||^2 using matrix operations, thus:

||x_i - x_j||^2 = <x_i - x_j, x_i - x_j> 
                = <x_i, x_i> - <x_i, x_j> - <x_j, x_i> + <x_j, x_j>
                = ||x_i||^2 - 2<x_i, x_j> + ||x_j||^2

and this is exactly what is implemented

First they take square of your data, as ||x_i||^2 = SUM_a x_i_a^2

nms = sum(X'.^2);

next they use multiplication with vector of ones to compute the sum opertion getting

nms'*ones(1,n)

which is vector of ||x_i||^2's, and analogously vector of ||x_j||^2's is

ones(n,1)*nms

and finally they compose using decomposition I wrote before, thus

-nms'*ones(1,n) -ones(n,1)*nms + 2*X*X')

is just a matrix A_ij = - ||x_i - x_j ||^2

In your case, you want to have division by 2sigma^2, thus just put it under the exp, after taking previous arugment in brackets, like

Ks = exp(-(nms'*ones(1,n) -ones(n,1)*nms + 2*X*X')/(2*sigma^2));