Conditional Replace for Custom Loss Function Constraint? TypeError: 'Tensor' object does not support item assignment

Question

I have a model that returns two values per sample in a batch where the first value must be >=0 and the second must be <=0. My first attempt at a custom error function follows:

def maxMinDiffLossConstrained(y_true, y_pred, sample_weight=None):
 
    y_pred_shape = list(y_pred.shape) # y_pred returns a tuple, we want a list so we can construct another tensor 
    #y_pred_data = y_pred.numpy() # get the tensor data as a numpy array 
        
    y_pred[y_pred[:,0] < 0, 0] = -10            
    y_pred[y_pred[:,1] > 0, 1] = 10            
        
    y_mod = tf.reshape(y_pred, y_pred_shape)   
   
    mse = MeanSquaredError()
    loss = mse(y_true, y_mod)

    return loss

Which is where I got the error

TypeError: 'Tensor' object does not support item assignment

As you can see I tried using y_pred.numpy() but the object but type(y_pred) is <class 'tensorflow.python.framework.ops.Tensor'> and returns the error:

AttributeError: 'Tensor' object has no attribute 'numpy'

I have checked a number of other posts. This one suggests unstacking and restacking

TypeError: 'Tensor' object does not support item assignment in TensorFlow

This next longer article was helpful, it suggested decomposing based on conditions and then remaking the tensor. I am still working towards a solution with this approach:

https://towardsdatascience.com/how-to-replace-values-by-index-in-a-tensor-with-tensorflow-2-0-510994fe6c5f

Unfortunately, since the dimensions of my Tensor are (batch_size, 2) I still have not figured out a solution that works. My difficulty is that, after I get the True/False vector for dim1 (>=0) and dim2 (<=0) with the following code:

maxes_to_remove = y_pred[:,0] < 0 
mins_to_remove = y_pred[:,1] > 0

idx_maxes = tf.where(maxes_to_remove == True)
idx_mins = tf.where(mins_to_remove == True)

I am not sure how to replace only the values in the 1st dimension that need to be replaced and only the values in the second that need to be replaced.

As a side note, i also ran into this issue but was able to solve it.

Answer 1

I believe tensors in tf are immutable , so you cannot assign values to indices in the array. Two workarounds:

1. Define a new tensor that is created using the original tensor and some additional operations. I recommend using tf.keras.backend.switch() , which allows you to assign values to a new tensor from one of two input tensors (which for you will be a tensor of -10 and a tensor of 10) based on the tensor of boolean values (for you this will be perhaps idx_maxes?)

2. Convert to numpy arrays, change as necessary, and convert back to tensor. Note that this only works if you are in eager execution mode. If you are in graph mode, intuitively converting to numpy is a no-go because numpy arrays do not support graph computation, so the chain of operations would be lost during conversion.