Using my own kernel in libsvm

Question

I am currently developing my own kernel to use for classification and want to include it into libsvm, replacing the standard kernels that libsvm offers.

I however am not 100% sure how to do this, and obviously do not want to make any mistakes. Beware, that my c++ is not very good. I found the following on the libsvm faq-page:

Q: I would like to use my own kernel. Any example? In svm.cpp, there are two subroutines for kernel evaluations: k_function() and kernel_function(). Which one should I modify ? An example is "LIBSVM for string data" in LIBSVM Tools.

The reason why we have two functions is as follows. For the RBF kernel exp(-g |xi - xj|^2), if we calculate xi - xj first and then the norm square, there are 3n operations. Thus we consider exp(-g (|xi|^2 - 2dot(xi,xj) +|xj|^2)) and by calculating all |xi|^2 in the beginning, the number of operations is reduced to 2n. This is for the training. For prediction we cannot do this so a regular subroutine using that 3n operations is needed. The easiest way to have your own kernel is to put the same code in these two subroutines by replacing any kernel.

Hence, I was trying to find the two subroutinges k_function() and kernel_function(). The former I found with the following signature in svm.cpp:

double Kernel::k_function(const svm_node *x, const svm_node *y,
              const svm_parameter& param)

Am I correct, that x and y each store one observation (=row) of my feature matrix in an array and I need to return the kernel value k(x,y)?

The function kernel_function() on the other hand I was not able to find at all. There is a pointer in the Kernel class with that name and the following declaration

double (Kernel::*kernel_function)(int i, int j) const; 

which is set in the Kernel constructor. What are i and j in that case? I suppose I need to set this pointer as well?

Once I overwrote Kernel::k_function and Kernel::*kernel_function I'd be finished, and libsvm would use my kernel to compare two observations?

Thank you!

Answer 1

You don't have to break into the code of LIBSVM to use your own kernel, you can use the pre-computed kernel option (ie, -t 4 training_set_file).

Thus, you can compute the kernel matrix externally as it suits you, store the values in a file and load the pre-computed kernel to LIBSVM. There's an explanation accompanied with an example of how to do this in the README file that you can find in LIBSVM tar ball (see in the Precomputed Kernels section line 236 ).