批量归一化层和演化归一化激活层有什么用

Question

When to decide that we need a batch or evolving layer and how do we decide it? I am currently using PyTorch and I want to understand how I can decide which layer to add?

Answer 1

General answer here is to try all of them and select the one, which performs best on the validation.

As for EvoNorm from this paper , it depends on your problem. Authors tested the new layer on classification problem with limited set of models. For image synthesis results weren't as good as for classification.

In my opinion, batchnorm is a good starting point to construct baseline solution, because it is time tested, and then try more advanced things.