XOR 神经网络误差不断收敛到 0.5

Question

I just started attempting to make my own neural network in c# by copying a simple one I found written in python and the output always ends at 0.5. I looked at all other questions posted about this same problem on Google and none of the solutions were applicable. I have no idea why this is happening. Here's what my code looks like right now:

public class NeuralNetwork
        {
            private double[,] w;

            public NeuralNetwork()
            {
                w = random(2, 1);
            }

            public double[,] getWeights()
            {
                return w;
            }

            public void train(double[,] x, double[,] y, int iterations)
            {
                for (int i = 0; i < iterations; i++)
                {
                    double[,] output = think(x);
                    double[,] error = Mat.subtract(Mat.transpose(y), output);
                    double[,] adjust = dot(Mat.transpose(x), Mat.multiply(error, doSigDeriv(output)));

                    w = Mat.add(w, adjust);
                }
            }

            public double[,] think(double[,] inputs)
            {
                return doSig(dot(inputs, w));
            }

            private double[,] dot(double[,] a, double[,] b)
            {
                double[,] dot = new double[a.GetLength(0), b.GetLength(1)];

                for (int i = 0; i < a.GetLength(0); i++)
                {
                    for (int j = 0; j < b.GetLength(1); j++)
                    {
                        for (int k = 0; k < b.GetLength(0); k++)
                            dot[i, j] += a[i, k] * b[k, j];
                    }
                }
                return dot;
            }

            private double[,] random(int w, int h)
            {
                double[,] a = new double[w, h];
                for (int i = 0; i < w; i++)
                {
                    for (int j = 0; j < h; j++)
                    {
                        a[i, j] = 2 * (double)rand.NextDouble() - 1;
                    }
                }
                return a;
            }

            private double[,] doSig(double[,] a)
            {
                double[,] b = new double[a.GetLength(0), a.GetLength(1)];
                for (int i = 0; i < a.GetLength(0); i++)
                {
                    for (int j = 0; j < a.GetLength(1); j++)
                    {
                        b[i, j] = sig(a[i, j]);
                    }
                }
                return b;
            }

            private double[,] doSigDeriv(double[,] a)
            {
                double[,] b = new double[a.GetLength(0), a.GetLength(1)];
                for (int i = 0; i < a.GetLength(0); i++)
                {
                    for (int j = 0; j < a.GetLength(1); j++)
                    {
                        b[i, j] = a[i, j] * (1 - a[i, j]);
                    }
                }
                return b;
                //return Mat.multiply(a, Mat.subtract(one(a), a));
            }

            private double sig(double x)
            {
               return 1 / (double)(1 + Math.Exp(-x));
            }
        }

x is the array I used for inputs and y for outputs, and Mat is just a class I made to handle the matrix operations. When I tested it, I changed the iterations to 10000 and 50000 and got the same result. Any insight is greatly appreciated, thanks.

EDIT: I just realized that when it's supposed to be 0, it's -0.5 and 1 is 0.5.

Answer 1

Of course the comment of user219279 is correct: NN better run in GPU/Tensor nowadays, which is much faster when you are going to scale up your network. For XOR back propagation and study however, you won't need performance.

An output value of 0.5 is very common and called "local minimum". It happens when your network training cannot converge, for some reason. Weights reach giant +/- values instead of a balance with small values and 0/1 outputs.

The error I see above: there are not enough weights. You allocate your weights as a 2x1 array, connecting input output. You can't train a XOR outcome net with only 2 weights. You need at least 5 weights. See for more info on a suitable XOR architecture http://mnemstudio.org/neural-networks-multilayer-perceptrons.htm