放宽线性约束？

Question

When we need to optimize a function on the positive real half-line, and we only have non-constraints optimization routines, we use y = exp(x) , or y = x^2 to map to the real line and still optimize on the log or the (signed) square root of the variable.

Can we do something similar for linear constraints, of the form Ax = b where, for xa d-dimensional vector, A is a (N,n) -shaped matrix and b is a vector of length N , defining the constraints ?

Answer 1

While, as Ervin Kalvelaglan says this is not always a good idea, here is one way to do it. Suppose we take the SVD of A, getting

A = U*S*V'
where if A is n x m
U is nxn orthogonal,
S is nxm, zero off the main diagonal, 
V is mxm orthogonal

Computing the SVD is not a trivial computation.

We first zero out the elements of S which we think are non-zero just due to noise -- which can be a slightly delicate thing to do.

Then we can find one solution x~ to

A*x = b 

as

x~ = V*pinv(S)*U'*b

(where pinv(S) is the pseudo inverse of S, ie replace the non zero elements of the diagonal by their multiplicative inverses)

Note that x~ is a least squares solution to the constraints, so we need to check that it is close enough to being a real solution, ie that Ax~ is close enough to b -- another somewhat delicate thing. If x~ doesn't satisfy the constraints closely enough you should give up: if the constraints have no solution neither does the optimisation.

Any other solution to the constraints can be written x = x~ + sum c[i]*V[i] where the V[i] are the columns of V corresponding to entries of S that are (now) zero. Here the c[i] are arbitrary constants. So we can change variables to using the c[] in the optimisation, and the constraints will be automatically satisfied. However this change of variables could be somewhat irksome!