Neural network example to classify multi-dimensional features into two sets

Question

I'm looking for a good source code example of a supervised neural.network which accepts more than two features (unlike most XY-examples) and classifies data into two sets. From what I've read, a Support Vector Machine (SVM) might be a solution?

All the classifying examples I have found are two-dimensional. Here are a few:

• Single layer peceptron
• SVM (Support Vector Machine) in CSharp
• SVM.NET - really nice, but input still two-dimensional.

I'm trying to distinguish rare events from a number of inputs which are normally stable. Features are key-value pairs where the value can usually discretized as a small number. Available training data for the first category is huge, but with only few training sets for the second category, if that makes a difference.

Example Training Set

Category A

[2, 1, 0, 1, 4, 3] -> A  
[1, 1, 2, 3, 3, 0] -> A
[0, 0, 1, 3, 2, 0] -> A

Category B

[0, 4, 4, 4, 4, 3] -> B

Classifying Example

[1, 3, 4, 4, 4, 0] -> ??? (probably B)

A confidence rating, eg. "85% certain of B", would be helpful in distinguishing a threshold for a rare event.

Is a neural.network the best solution and are there any .NET libraries with this built-in?

Answer 1

In reality, all these machine learning techniques have their pros and cons. In using NN (single layer perceptron), you need to consider if you have enough training data. Technically speaking, you need to be able to cover all cells inside the dimensions to have a good result.

SVM on the other hand, tries to find a border separating your data points so if you have gaps in the areas which are not close to this border, it is fine.

There are 5-6 classifiers around +/- boosting and to be honest, it seems that most of the time type of the classifier is chosen subjectively. On the other hand, some people use multiple classifiers and compare the result.

With OpenCV, it is so easy to pluggin a different classifier so you are on right track for it. I used OpenCV in C++ with NN classifiers for my project and result was very good:

http://www.springerlink.com/content/j0615767m36m0614/

Answer 2

SVM is n-dimensional - it's just that the EXAMPLES are usually 2D, since once you get to any more than 3 the solution doesn't really fit into 2D illustrations anymore.

It only has two output classes (usually Good and Bad), but it has as many features as you like. That's why the line splitting your two SVM classes is called a 'hyperplane', since it exists in multi-dimensional space - one dimension for each feature.