What is trainable parameter in tensorflow?

Question

tf.compat.v1.layers.batch_normalization takes trainable as an input. The documentation says:

Boolean, if True also add variables to the graph collection GraphKeys.TRAINABLE_VARIABLES (see tf.Variable).

I think only scaling factor (gamma) and offset (beta) should be added to trainable variables and I am skeptical if even moving averages will get added to GraphKeys.TRAINABLE_VARIABLES. Can somebody tell me how trainable input is influencing the behavior of batch_normalization

Answer 1

First of all, this function is deprecated and should not be used.

trainable arguments means that scaling factor (gamma) and offset (beta) will be trainable and it's true by default.

When it comes to moving averages, those are not trainable , they are only updated after each batch pass, those are not parameters ( tf.Variable objects).

Please notice, you can set trainable to false, in such case, if beta and gamma are set to defaults (zero and one respectively), they won't affect the moving averages. You can turn them off by issuing center (for beta ) or scale (for gamma ).