[英]What activation layers learn?
I am trying to figure out what CNN architecture after every activation layers.我试图弄清楚每个激活层之后的 CNN 架构。 Therefore, I have written a code to visualize some activation layers in my model.
因此,我编写了一个代码来可视化我模型中的一些激活层。 I used LeakyReLU as my activation layer.
我使用 LeakyReLU 作为我的激活层。 This is the figure LeakyRelu after Conv2d + BatchNorm
这是Conv2d + BatchNorm后的LeakyRelu图
As can be seen from the figure, there are quite purple frames, which shows nothing.从图中可以看出,有相当紫色的边框,没有任何显示。 So my question is what does it mean.
所以我的问题是它是什么意思。 Does my model learn anything?
我的模型能学到什么吗?
Generally speaking, activation layers (AL) don't learn.一般来说,激活层 (AL) 不会学习。 The purpose of AL is to add non-linearity into the model, hence they usually apply a certain, fixed, function regardless of the data, without adapting with the data.
AL 的目的是在模型中添加非线性,因此它们通常应用特定的、固定的函数而不考虑数据,而不是适应数据。 As an example:
举个例子:
I tried to simplify the math, so pardon my inaccuracies.我试图简化数学,所以请原谅我的不准确之处。 As a closure, your purple frames are probably filters that didn't learn just yet, train the model to convergence and unless your model is highly bloated (too big for your data) your will see 'structures' in your filters.
作为一个闭包,您的紫色框架可能是尚未学习的过滤器,训练模型以使其收敛,除非您的模型非常臃肿(对于您的数据来说太大),否则您将在过滤器中看到“结构”。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.