[英]Keras custom softmax layer: Is it possible to have output neurons set to 0 in the output of a softmax layer based on zeros as data in an input layer?
I have a neural network with 10 output neurons in the last layer using softmax activation. 我有一个使用softmax激活的神经网络,在最后一层有10个输出神经元。 I also know exactly that based on the input values, certain neurons in the output layer shall have 0 values.
我也完全知道,基于输入值,输出层中的某些神经元应具有0值。 So I have a special input layer of 10 neurons, each of them being either 0 or 1.
因此,我有一个特殊的输入层,其中包含10个神经元,每个神经元为0或1。
Would it be somehow possible to force let's say the output neuron no. 强迫输出神经元为no。 3 to have value = 0 if the input neuron no 3 is also 0?
如果输入的第3个神经元也为0,则3的值等于0?
action_input = Input(shape=(10,), name='action_input')
...
x = Dense(10, kernel_initializer = RandomNormal(),bias_initializer = RandomNormal() )(x)
x = Activation('softmax')(x)
I know that there is a method via which I can mask out the results of the output layer OUTSIDE the neural network, and have all non zero related outputs reshaped (in order to have a total sum of 1). 我知道有一种方法可以掩盖神经网络外输出层的结果,并对所有非零相关输出进行整形(以使总和为1)。 But I would like to solve this issue within the network and use it during the training of the network, too.
但是我想在网络中解决这个问题,并在网络训练中使用它。 Shall I use a custom layer for this?
我应该为此使用自定义图层吗?
You can use a Lambda
layer and K.switch
to check for zero values in the input and mask them in the output: 您可以使用
Lambda
层和K.switch
来检查输入中的零值,并在输出中将其屏蔽:
from keras import backend as K
inp = Input((5,))
soft_out = Dense(5, activation='softmax')(inp)
out = Lambda(lambda x: K.switch(x[0], x[1], K.zeros_like(x[1])))([inp, soft_out])
model = Model(inp, out)
model.predict(np.array([[0, 3, 0, 2, 0]]))
# array([[0., 0.35963967, 0., 0.47805876, 0.]], dtype=float32)
However, as you can see the sum of outputs are no longer one. 但是,您可以看到输出的总和不再是一个。 If you want the sum to be one, you can rescale the values:
如果希望总和为1,则可以重新缩放值:
def mask_output(x):
inp, soft_out = x
y = K.switch(inp, soft_out, K.zeros_like(inp))
y /= K.sum(y, axis=-1)
return y
# ...
out = Lambda(mask_output)([inp, soft_out])
At the end I came up with this code: 最后,我想到了以下代码:
from keras import backend as K
import tensorflow as tf
def mask_output2(x):
inp, soft_out = x
# add a very small value in order to avoid having 0 everywhere
c = K.constant(0.0000001, dtype='float32', shape=(32, 13))
y = soft_out + c
y = Lambda(lambda x: K.switch(K.equal(x[0],0), x[1], K.zeros_like(x[1])))([inp, soft_out])
y_sum = K.sum(y, axis=-1)
y_sum_corrected = Lambda(lambda x: K.switch(K.equal(x[0],0), K.ones_like(x[0]), x[0] ))([y_sum])
y_sum_corrected = tf.divide(1,y_sum_corrected)
y = tf.einsum('ij,i->ij', y, y_sum_corrected)
return y
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.