[英]Using a custom step activation function in Keras results in “An operation has `None` for gradient.” error. How to resolve this?
I am building auto-encoder and I want to encode my values into a logical matrix. 我正在构建自动编码器,我想将我的值编码为逻辑矩阵。 However, when I'm using my custom step activation function in one of the intermediate layers (all other layers are using 'relu'), keras raises this error:
但是,当我在一个中间层(所有其他层都使用“ relu”)中使用我的自定义步骤激活功能时,keras会引发以下错误:
An operation has `None` for gradient.
I've tried using hard-sigmoid function, but it doesn't fit my problem, because it still produces intermediate values, when I only need binary. 我尝试使用硬S型函数,但是它不适合我的问题,因为当我只需要二进制时,它仍然会产生中间值。 I am aware, that at most points my function has no gradient, but is it possible to use some other function for gradient calculation and still use step function for accuracy and loss calculations?
我知道,我的函数在大多数时候都没有梯度,但是是否可以使用其他函数进行梯度计算,而仍使用步进函数进行精度和损耗计算?
My activation function: 我的激活功能:
def binary_activation(x):
ones = tf.ones(tf.shape(x), dtype=x.dtype.base_dtype)
zeros = tf.zeros(tf.shape(x), dtype=x.dtype.base_dtype)
return keras.backend.switch(x > 0.5, ones, zeros)
I expect to be able to use binary step activation function to train the network and then use it as a typical auto-encoder. 我希望能够使用二进制步进激活功能来训练网络,然后将其用作典型的自动编码器。 Something simmilar to binary feature map used in this paper .
本文使用的类似二元特征映射的东西。
As mentioned here , you could use tf.custom_gradient to define a "back-propagatable" gradient for your activation function. 如前所述在这里 ,你可以使用tf.custom_gradient定义“后繁殖的”梯度为您激活功能。
Perhaps something like: 也许像这样:
@tf.custom_gradient
def binary_activation(x):
ones = tf.ones(tf.shape(x), dtype=x.dtype.base_dtype)
zeros = tf.zeros(tf.shape(x), dtype=x.dtype.base_dtype)
def grad(dy):
return ... # TODO define gradient
return keras.backend.switch(x > 0.5, ones, zeros), grad
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.