Matrix must be positive definite (Math.Net C# library)

Question

I'm encountering a strange issue with the Math.Net Numerics library for C#. My code worked perfectly fine until recently (nothing has changed as far as I can tell) but I'm now getting the error message from the title at the line where it tries to calculate the multiple regression.

Each list has 493 double values so does anyone know what I can do to fix these issues?

Vector<double> vectorArrayBuy = CreateVector.Dense(listMRInfoBuy.ElementAt(0).OutputBuy.ToArray());

var matrixArrayBuy = CreateMatrix.DenseOfColumnArrays(listMRInfoBuy.ElementAt(0).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(1).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(2).ListValuesBuy.ToArray(),
                                listMRInfoBuy.ElementAt(3).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(4).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(5).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(6).ListValuesBuy.ToArray(),
                                listMRInfoBuy.ElementAt(7).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(8).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(9).ListValuesBuy.ToArray(), listMRInfoBuy.ElementAt(10).ListValuesBuy.ToArray(),
                                listMRInfoBuy.ElementAt(11).ListValuesBuy.ToArray());

var itemsBuy = MultipleRegression.NormalEquations(matrixArrayBuy, vectorArrayBuy);

Answer 1

"Matrix not positive definite" probably means that you have fewer than n independent equations, which in turn means you don't have n independent data, which probably means your data are defective in some way (eg they were read in incorrect and they're actually all the same or something like that).

Perhaps you can edit your question to show what are the data you are working with. Maybe you have fewer than n data to start with.

Answer 2

Just adding a solution for those (like me) who do not remember linear algebra or advanced stats from your school days.

1. If you haven't already, apply the "Analysis ToolPak" Add-In in Excel
2. Paste your independent and dependent variables into a worksheet
3. Go to Data -> Data Analysis -> Regression
4. Provide the ranges it asks for and run the regression
5. In the regression results you'll find that one or more p-values or t-stats return an a div/0 or NUM error.
6. Remove those independent variables from your call to the MathNet regression and run again.

This should fix it.

I then went on to add an iterative try and catch that would remove the independent variables, based upon the specific circumstance, and run it again.

IHTH

Answer 3

I fixed this issue by switching on the fly for the different equations to see which one returned the correct answers and didn't throw this exception. Here is my solution to this problem which I hope helps someone else out.

public Vector<double> CalculateWithQR(Matrix<double> x, Vector<double> y)
    {
        Vector<double> result = null;

        try
        {
            result = MultipleRegression.QR(x, y);

            // check for NaN and infinity
            for (int i = 0; i < result.Count; i++)
            {
                var value = result.ElementAt(i);

                if (Double.IsNaN(value) || Double.IsInfinity(value))
                {
                    return null;
                }
            }
        }
        catch (Exception ex)
        {
        }

        return result;
    }

    public Vector<double> CalculateWithNormal(Matrix<double> x, Vector<double> y)
    {
        Vector<double> result = null;

        try
        {
            result = MultipleRegression.NormalEquations(x, y);

            // check for NaN and infinity
            for (int i = 0; i < result.Count; i++)
            {
                var value = result.ElementAt(i);

                if (Double.IsNaN(value) || Double.IsInfinity(value))
                {
                    return null;
                }
            }
        }
        catch (Exception ex)
        {
        }

        return result;
    }

    public Vector<double> CalculateWithSVD(Matrix<double> x, Vector<double> y)
    {
        Vector<double> result = null;

        try
        {
            result = MultipleRegression.Svd(x, y);

            // check for NaN and infinity
            for (int i = 0; i < result.Count; i++)
            {
                var value = result.ElementAt(i);

                if (Double.IsNaN(value) || Double.IsInfinity(value))
                {
                    return null;
                }
            }
        }
        catch (Exception ex)
        {
        }

        return result;
    }

    public Vector<double> FindBestMRSolution(Matrix<double> x, Vector<double> y)
    {
        Vector<double> result = null;

        try
        {
            result = CalculateWithNormal(x, y);

            if (result != null)
            {
                return result;
            }
            else
            {
                result = CalculateWithSVD(x, y);

                if (result != null)
                {
                    return result;
                }
                else
                {
                    result = CalculateWithQR(x, y);

                    if (result != null)
                    {
                        return result;
                    }
                }
            }
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
            Console.WriteLine(ex.StackTrace);
        }

        return result;
    }