C# Math.Net Matrix dimensions must agree

Question

I was trying to translate this python code for a Neural Network https://gist.github.com/miloharper/c5db6590f26d99ab2670#file-main-py in C#. I'm using the Math.Net Numerics for the matrixes and here is the code I've made so far in C#

using System;
using MathNet.Numerics.LinearAlgebra;
using MathNet.Numerics;
using MathNet.Numerics.LinearAlgebra.Double;

namespace NeuralNetwork
{
    class Program
    {
        static void Main(string[] args)
        {
            NeuralNetwork NN = new NeuralNetwork();

            Console.WriteLine("Random starting synaptic weights: ");
            Console.WriteLine(NN.SynapticWeights);

            Matrix<double> TrainingSetInput = DenseMatrix.OfArray(new double[,] { { 0, 0, 1 }, { 1, 1, 1 }, { 1, 0, 1 }, { 0, 1, 1 } });
            Matrix<double> TrainingSetOutput = DenseMatrix.OfArray(new double[,] { { 0, 1, 1, 0 } }).Transpose();

            NN.Train(TrainingSetInput, TrainingSetOutput, 10000);

            Console.WriteLine("New synaptic weights after training: ");
            Console.WriteLine(NN.SynapticWeights);

            Console.WriteLine("Considering new situation {1, 0, 0} -> ?: ");
            Console.WriteLine(NN.Think(DenseMatrix.OfArray(new double[,] { { 1, 0, 0 } })));

            Console.ReadLine();
        }
    }

    class NeuralNetwork
    {
        private Matrix<double> _SynapticWeights;

        public NeuralNetwork()
        {
            _SynapticWeights = 2 * Matrix<double>.Build.Random(3, 1) - 1;
        }

        private Matrix<double> Sigmoid(Matrix<double> Input)
        {
            return 1 / (1 + Matrix<double>.Exp(-Input));
        }

        private Matrix<double> SigmoidDerivative(Matrix<double> Input)
        {
            return Input * (1 - Input); //NEW Exception here
        }

        public Matrix<double> Think(Matrix<double> Input)
        {
            return Sigmoid((Input * _SynapticWeights)); //Exception here (Solved)
        }

        public void Train(Matrix<double> TrainingInput, Matrix<double> TrainingOutput, int TrainingIterations)
        {
            for (int Index = 0; Index < TrainingIterations; Index++)
            {
                Matrix<double> Output = Think(TrainingInput);
                Matrix<double> Error = TrainingOutput - Output;
                Matrix<double> Adjustment = Matrix<double>.op_DotMultiply(TrainingInput.Transpose(), Error * SigmoidDerivative(Output));

                _SynapticWeights += Adjustment;
            }
        }

        public Matrix<double> SynapticWeights { get { return _SynapticWeights; } set { _SynapticWeights = value; } }
    }
}

When I execute it, it shows an exception on line 53 (there is a comment at that line in the code). It says:

Matrix dimensions must agree: op1 is 4x3, op2 is 3x1

Did I copy it wrong or is it a problem with the MAth.Net library?

Thanks in advance :D

Answer 1

As far as I can see, the problem not in the library code. You are trying to perform an element-wise matrix multiplication with the use of op_DotMultiply function (line 53). In this case it is obvious from the error message that matrices cannot be multiplied due to difference in their size: the first one is 4x3, the second one is 3x1.

I can suggest to look at the size of these matrices: TrainingSetInput and _SynapticWeights (look at function Train , you are calling Think inside of it with the wrong sized matrices). Check if they are generated correctly. And also think if you really need an element-wise matrix multiplication or a usual multiplication. If you need more info about matrix multiplications, you can use these links:

Element-wise: https://en.wikipedia.org/wiki/Hadamard_product_(matrices)

Usual: https://en.wikipedia.org/wiki/Matrix_multiplication