具有正映射的线性回归

Question

I am using Tensorflow for my research. Recently, I am trying to implement a linear regression in Tensorflow using least squares with positive mapping K, ie, min ||y-Kx|| such that all elements of K are positive values, where x and y are vectors which are mapped through the matrix K. Basically, I want to enforce the trainable variables K to be positive when implementing it in Tensorflow.

I've searching for a good way to implement it in Tensorflow. Some articles suggest to use tf.clip_by_value, but it is likely to have bunch of zero entries in K. I've found tf.Variable has "constraint" argument which seems to do projection of solution onto a constrained space, but I haven't been able to find a good example to follow for my case.

I would appreciate any suggestion to enforce positive values for trainable variables in Tensorflow. Thank you for your help in advance.

Answer 1

Note that constraint :

constraint function that is applied after the update of the optimizer . (eg used to implement norm constraints or value constraints for layer weights)

This means that, following variables all have the same value before update of the optimizer:

myVar1 = tf.Variable([1,-1,2],constraint=lambda x: tf.clip_by_value(x, 0, np.infty))
myVar2 = tf.Variable([1,-1,2],constraint=lambda x: tf.abs(x))
myVar3 = tf.Variable([1,-1,2],constraint=lambda x: tf.clip_by_norm(x,axis=-1))

Another things worth to mention is that, you can also define your own constraint as well:

def my_constrain(x):
    x = tf.abs(x)
    return tf.clip_by_value(x, 0, 0.5)

a = tf.get_variable(name='a', initializer=0., constraint=lambda x: my_constrain(x))

---

Another suggestion that you can consider is regularization for a specific parameter. For example, adaptive Lasso ( H.Zou, JASA 2006, Vol. 101, No. 476 ) achieves consistency in parameter estimates by using individual lambda for each variable.