如何反向批量归一化

Question

In a regression network, I would like to use batch normalization on the objective y to obtain y_norm to fit. Because y_norm is well distributed.

In testing stage after training, I need to "undo" a batch normalization on the predicted y_norm . Is there any elegant way in tensorflow/keras in which I can construct an "undo" layer from the origin BN layer?

Answer 1

I have found that this layer has 2 methods: inverse() which performs the normalization and forward() which performs the de-normalization. So, in training you should use inverse and in inference the forward method.