澄清批量标准化如何在 Tensorflow 上工作

Question

To better understand how the BN works, I have decided to code my batch normalization and compare it to the TF implementation. I made a simple example (image attached). I computed the mean and variance for each channel across the batches, as illustrated in the image. After that, I normalized the data using the formulation presented in the BN paper , subtracted by the mean, and divided by the root of the variance (I am not using the scaled factor nor the shift). Then, I used the function tf.keras.layers.BatchNormalization(center=False, scale=False) from TF. The results were completely different.

I would like some help to clarify how the batch normalization (BN) is performed on TensorFlow (TF) and if anyone has some insight into why the results were so different from TF.

Thank you.

Answer 1

you can read the TF's documentation and implementation here

• https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization
• https://github.com/keras-team/keras/blob/bf6859b0dadc34c39ac26e4e25ee434e756e8914/keras/layers/normalization/batch_normalization.py