Writing a custom loss function element by element for Keras

Question

I am new to machine learning, python and tensorflow. I am used to code in C++ or C# and it is difficult for me to use tf.backend. I am trying to write a custom loss function for an LSTM network that tries to predict if the next element of a time series will be positive or negative. My code runs nicely with the binary_crossentropy loss function. I want now to improve my network having a loss function that adds the value of the next time series element if the predicted probability is greater than 0.5 and substracts it if the prob is less or equal to 0.5. I tried something like this:

def customLossFunction(y_true, y_pred):
    temp = 0.0
    for i in range(0, len(y_true)):
        if(y_pred[i] > 0):
            temp += y_true[i]
        else:
            temp -= y_true[i]
    return temp

Obviously, dimensions are wrong but since I cannot step into my function while debugging, it is very hard to get a grasp of dimensions here. Can you please tell me if I can use an element-by-element function? If yes, how? And if not, could you help me with tf.backend? Thanks a lot

Answer 1

From keras backend functions, you have the function greater that you can use:

import keras.backend as K

def customLossFunction(yTrue,yPred)

    greater = K.greater(yPred,0.5)
    greater = K.cast(greater,K.floatx()) #has zeros and ones
    multiply = (2*greater) - 1 #has -1 and 1

    modifiedTrue = multiply * yTrue

    #here, it's important to know which dimension you want to sum
    return K.sum(modifiedTrue, axis=?)

The axis parameter should be used according to what you want to sum.

axis=0 -> batch or sample dimension (number of sequences)     
axis=1 -> time steps dimension (if you're using return_sequences = True until the end)     
axis=2 -> predictions for each step 

Now, if you have only a 2D target:

axis=0 -> batch or sample dimension (number of sequences)
axis=1 -> predictions for each sequence

If you simply want to sum everything for every sequence, then just don't put the axis parameter.

Important note about this function:

Since it contains only values from yTrue , it cannot backpropagate to change the weights. This will lead to a "none values not supported" error or something very similar.

Although yPred (the one that is connected to the model's weights) is used in the function, it's used only for getting a true x false condition, which is not differentiable.