在 Tensorflow/Keras 中将 NN 输出的最大值限制为正数

Question

I have a neural network that outputs a number of outputs num_out .

I know that if I want all outputs to be positive, I could apply a relu activation function (or others) on the output layer.

However, my goal is to ensure that only the maximum value among all num_out outputs is positive. I cannot find the way to enforce that.

Answer 1

One way to solve your problem is to use tf.where and tf.reduce_max :

import tensorflow as tf
x = tf.constant([
                 [-1, -2, -3], 
                 [-4, -5, -6]
                ])

max_val = tf.reduce_max(x, keepdims=True)
new_max_val = tf.where(tf.greater(max_val, 0), max_val, tf.math.negative(max_val))
result = tf.where(tf.equal(x, max_val), tf.ones(tf.shape(x), dtype=x.dtype) * new_max_val, x)
 
print('Max value: ', max_val)
print('New Max value: ', new_max_val)
print('Result: ', result)

Max value:  tf.Tensor([[-1]], shape=(1, 1), dtype=int32)
New Max value:  tf.Tensor([[1]], shape=(1, 1), dtype=int32)
Result:  tf.Tensor(
[[ 1 -2 -3]
 [-4 -5 -6]], shape=(2, 3), dtype=int32)

I first find the max value in tensor x and then convert it into a positive value if it is negative, otherwise it stays the same. Afterwards, I update the tensor x with the new value. If the highest value in your tensor is positive, then nothing changes:

x = tf.constant([
                 [-1, -2, -3], 
                 [ 4,  5,  6]
                ])
# -->

Max value:  tf.Tensor([[6]], shape=(1, 1), dtype=int32)
New Max value:  tf.Tensor([[6]], shape=(1, 1), dtype=int32)
Result:  tf.Tensor(
[[-1 -2 -3]
 [ 4  5  6]], shape=(2, 3), dtype=int32)

If you want to make sure that all other elements except the maximum value are negative, then change this line:

result = tf.where(tf.equal(x, max_val), tf.ones(tf.shape(x), dtype=x.dtype) * new_max_val, tf.abs(x)*-1)

Max value:  tf.Tensor([[6]], shape=(1, 1), dtype=int32)
New Max value:  tf.Tensor([[6]], shape=(1, 1), dtype=int32)
Result:  tf.Tensor(
[[-1 -2 -3]
 [-4 -5  6]], shape=(2, 3), dtype=int32)