如何使用另一个模型的输出作为参数来实现自定义Keras稳压器？

Question

I'm trying to replicate this article: https://arxiv.org/pdf/1705.08302.pdf

Basically, a fully convolutional network (FCN) does voxel level predictions on a patch of a image, then, this patch and its respective labels are passed through an autoencoder and then compared to evaluate the "global shape" of the predictions.

So the loss function (eq. (1) page 4) is a linear combination between the cross entropy from the FCN and euclidean distance loss from the autoencoder.

Problem:

I have a working FCN and a working autoencoder, my problem has been implementing this loss function in Keras/tensorflow. So, how can I do that?

This is what I tried so far (without third term of equation) but gives wrong results:

def euclidean_distance_loss(y_true, y_pred):
    from keras import backend as K
return K.sqrt(K.sum(K.square(y_pred - y_true)))

def ACNN_loss(l1, autoencoder):
    from keras import backend as K

    def loss(y_true, y_pred):
        ae_seg = autoencoder(y_pred)
        ae_gt = autoencoder(y_true)

        Lhe = K.sqrt(K.sum(K.square(ae_seg - ae_gt)))

        Lx = K.binary_crossentropy(y_true, y_pred)

        return Lx + (l1 * Lhe)

return loss

l1 = 0.01

ae_path = #path of my autoencoder model and its weights
autoencoder = keras.models.load_model(os.path.join(ae_path,'model.h5'), custom_objects={'euclidean_distance_loss': euclidean_distance_loss})

autoencoder.load_weights(os.path.join(ae_path,'weigths.h5'))


model.compile(loss = ACNN_loss(l1, autoencoder),
              optimizer = keras.optimizers.Adam(lr=0.0003, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0),
              metrics= ['accuracy', keras.metrics.binary_crossentropy]
              )

This is my first question so sorry if I messed up on any requirements. Thanks in advance

Answer 1

The square- root is unnecessary. Looking on the paper you attached, the loss doesn't contains a sqrt function. Ie the regularization term takes the squared-distance, instead of the norm-distance

Specifically, you should replace

Lhe = K.sqrt(K.sum(K.square(ae_seg - ae_gt)))

with

Lhe = K.sum(K.square(ae_seg - ae_gt))

In general, L2 regularization always takes the squared distance.