How to find least-mean-square error quadratic upper bound?

Question

I have some data of the form

x1[i], x2[i], x3[i], z[i] ,

where z[i] is an unknown deterministic function of x1[i], x2[i], and x3[i] . I would like to find a quadratic function u(x1, x2, x3)= a11*x1^2 + a22*x2^2 + a33*x3^2 + a12*x1*x2 + ... + a0 that overbounds the data, ie, u(x1[i], x2[i], x3[i]) >= z[i] for all i , and that minimizes the sum of the squared errors subject to the constraints.

Is there a computationally efficient solution approach in either Python or Matlab?

Answer 1

Your problem sounds like a quadratic programming problem with linear constraints. There are efficient algorithms to solve these and they are implemented in Matlab and Python as well; see quadprog and CVXOPT respectively.

Answer 2

There is very simple solution. Just use polynomial regression in Mathlab ( http://www.matrixlab-examples.com/polynomial-regression.html ). You will get a certain function P(x1[i],x2[i],x3[i]). 1. Then for each i compute expression Diff[i] = P(x1[i],x2[i],x3[i]) - z[i]. You will get some array Diff. 2. Select all the negative values. 3. Find the minimum value in Diff: M = Min(Diff). 4. The desired function is F(x1[i],x2[i],x3[i]) = P(x1[i],x2[i],x3[i]) + Abs(M), where Abs(M) - it's value excluding the sign of M.

But if you're not just limited to quadratic functions, you can vary the degree of the polynomial and eventually get a more precise solution.