Encog神经网络错误永不改变

Question

I'm getting started working with neural networks. I have adapted the XOR example provided for my own purposes, but when I run it the error never changes.

The function I'm trying to approximate takes 4 doubles and outputs 1 double, the 4 inputs are always positive, the output can be negative or positive (majority is positive). For starters I am using 50 records to train the data.

Working XOR (The error goes down with each iteration)

    public static double[][] XORInput = {
        new[] {0.0, 0.0},
        new[] {1.0, 0.0},
        new[] {0.0, 1.0},
        new[] {1.0, 1.0}
    };

    public static double[][] XORIdeal = {
        new[] {0.0},
        new[] {1.0},
        new[] {1.0},
        new[] {0.0}
    };

    BasicNetwork network = new BasicNetwork();
    network.AddLayer(new BasicLayer(null, true, 2));
    network.AddLayer(new BasicLayer(new ActivationSigmoid(), true, 3));
    network.AddLayer(new BasicLayer(new ActivationSigmoid(), false, 1));
    network.Structure.FinalizeStructure();
    network.Reset();

    IMLDataSet trainingData = new BasicMLDataSet(XORInput, XORIdeal);

    IMLTrain train = new ResilientPropagation(Network, trainingData);

Not Working (the error never goes down):

    BasicNetwork network = new BasicNetwork();
    network.AddLayer(new BasicLayer(null, true, 4));
    network.AddLayer(new BasicLayer(new ActivationSigmoid(), true, 6));
    network.AddLayer(new BasicLayer(new ActivationSigmoid(), false, 1));
    network.Structure.FinalizeStructure();
    network.Reset();

    IMLDataSet trainingData = new BasicMLDataSet(myInput, myExpectedOutput);

    IMLTrain train = new ResilientPropagation(Network, trainingData);

A few sample records of the training data:

    Input:          
    2.54, 3.15, 3.4, 1.73
    5.3, 1.78, 3.9, 2.04
    1.71, 5.4, 4.3, 2.26
    1.62, 6.4, 4, 1.89
    1.45, 8.4, 5.2, 2.14

    Output:
    5.59
    11.05
    6.89
    10.4
    -0.56

I believe that the problem is that activation function isn't firing. I thought it might be because ActivationSigmoid() is inappropriate for this problem, but I have tried ActivationTANH() with the exact same results.

Answer 1

The problem is that my values weren't being normalised.

To work with the activation functions all of your inputs and outputs must be between 1 and 0 (ActivationSigma) and -1 and 1 (ActivationTANH). You need some function to normalise your values to the range that they need to be.

This link was of great help to me:

http://www.heatonresearch.com/wiki/Range_Normalization