如何在 Keras model 中使用加权 MSE 作为损失 function？

Question

I am trying to use a custom loss function for calculating a weighted MSE in a regression taks (values in the task:-1,-0.5, 0, 0.5, 1, 1.5, 3 etc.). Here is my implementation of custom loss function:

import tensorflow
import tensorflow.keras.backend as kb

def weighted_mse(y, yhat):
    ind_losses = tensorflow.keras.losses.mean_squared_error(y, yhat)
    weights_ind = kb.map_fn(lambda yi: weight_dict[kb.get_value(yi)], y, dtype='float32')
    # average loss over weighted sum of the batch
    return tensorflow.math.divide(tensorflow.math.reduce_sum(tensorflow.math.multiply(ind_losses, weights_ind)), len(y))

I am running an example which is working:

weight_dict = {-1.0: 70.78125, 0.0: 1.7224334600760458, 0.5: 4.58502024291498, 1.0: 7.524916943521595, 1.5: 32.357142857142854, 2.0: 50.33333333333333, 2.5: 566.25, 3.0: 566.25}
y_true = tensorflow.convert_to_tensor([[0.5],[3]])
y_pred = tensorflow.convert_to_tensor([[0.5],[0]])

weighted_mse(y_true, y_pred)

But when inputted into my model, it throws the following error:

AttributeError: 'Tensor' object has no attribute '_numpy'

Here is how I use the custom loss function:

    model.compile(
    optimizer=opt,
    loss={
        "predicted_class": weighted_mse
    })

EDIT:

when changing weight_dict[kb.get_value(yi)] to weight_dict[float(yi)] I get the following error:

TypeError: float() argument must be a string or a number, not 'builtin_function_or_method'

Answer 1

What you want is basically the idea of sample weight. When using training API of Keras, alongside your data you can pass another array containing the weight for each sample which is used to determine the contribution of each sample in the loss function.

To use it, you can use sample_weight argument of fit method:

model.fit(X, y, sample_weight=X_weight, ...)

Note that X_weight should be an array of the same length as X (ie one weight value for each training sample). Further, if X is a tf.data.Dataset instance or a generator, this argument does not work and instead you need to pass the sample weight as the third element of the tuple returned by X .

Answer 2

This usually happens in an old version of tensorflow. There are 2 things you can try:

1. Add this line to the jupyter notebook when you are importing tensorflow like so:

import tensorflow as tf
tf.enable_eager_execution()

2. Upgrade tensorflow with the following command in prompt:

pip install tensorflow --upgrade

This is most probably because of eager execution. See the docs here for more info.