基于错误百分位数的Keras中的自定义加权MSE损失函数

Question

I'm new to Keras and neural networks in general. I'm trying to implement a custom loss function based on mean squared error for a multi-layer autoencoder to be used in anomaly detection. Basically the approach I'm going for is from here https://www.jstage.jst.go.jp/article/ipsjjip/27/0/27_335/_pdf

Unfortunately I don't have the reputation to post images as I'm also new to SO but the formula is on page 2, section 3 as Lprop

The intuition here is that I don't want the autoencoder to update weights for data points that return errors above the ap percentile of losses. This way it learns to reconstruct the inliers in the dataset while struggling with the outliers, hence detecting them as anomalous.

Here's some code I've tried and the compiled model

import keras.backend as K
c = 70.0
    def mean_squared_errorx(y_true, y_pred):
        es = K.square(y_pred - y_true)
        const = np.percentile(es, c)
        w = K.cast(K.less(const, K.mean(K.square(y_pred - y_true), axis=-1)), dtype = "float32")
        return w * K.mean(K.square(y_pred - y_true), axis=-1)

    #'mean_squared_error'
    autoencoder.compile(optimizer=adam, loss=mean_squared_errorx)
    autoencoder.fit(train, train,
                    epochs=num_epochs,
                    batch_size=round(len(train)/50),
                    shuffle=True,
                    validation_data=(train, train),
                    verbose = 0)
    encoded_d = encoder.predict(train)
    decoded_pred = decoder.predict(encoded_d)

The idea is to get the K.less to return a bool for each error, and then to convert it to a float to serve as a weight in the return statement. I know the np.percentile part probably won't work on a Tensor but don't know how else to accomplish the percentile ranking.

With that code I'm getting this error message

InvalidArgumentError: Incompatible shapes: [37,21] vs. [37]
     [[{{node loss_25/dense_104_loss/Less}}]]

where in this case the batch size is 37 and the number of features is 21. I appreciate any feedback on this or other parts of the code - thanks!

Answer 1

Found a potential workaround if anybody is working on something similar

import keras.backend as K
    def mean_squared_error_w(y_true, y_pred):
        mses = K.mean(K.square(y_pred - y_true), axis = -1)
        std_of_mses = K.std(mses)
        const = K.mean(mses, axis = -1) + (std_of_mses * 0.5)
        mask = K.cast(K.less(K.mean(K.square(y_pred - y_true), axis=-1), const), dtype = "float32")
        return mask * K.mean(K.square(y_pred - y_true), axis=-1)

I believe this will create a tensor of bools for all of the values where the error is larger than a threshold value, defined by the mean of the batch MSEs plus half a standard deviation (if the errors were normally distributed this should correspond to about the 70th percentile of the data as the cutoff). It converts the bools to the weights 0 or 1 as a mask which is then applied to the output MSE loss