[英]How to dropout entire hidden layer in a neural network?
I am trying to build a neural network in tensorflow 2.0.我正在尝试在 tensorflow 2.0 中构建神经网络。 There I want to dropout the whole hidden layer with a probability not any single node with a certain probability.在那里,我想以一定概率而不是任何单个节点的概率丢弃整个隐藏层。 Can anyone please tell me how to dropout the entire layer in tensorflow 2.0?谁能告诉我如何在 tensorflow 2.0 中退出整个层?
Use the noise_shape
argument of the Dropout
layer to be [1] * n_dim
of the input.使用Dropout
层的noise_shape
参数为输入的[1] * n_dim
。 Let's say the input tensor is 2D:假设输入张量是 2D:
import tensorflow as tf
x = tf.ones([3,5])
<tf.Tensor: shape=(3, 5), dtype=float32, numpy=
array([[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1.]], dtype=float32)>
noise_shape
should be [1, 1]
. noise_shape
应该是[1, 1]
。
tf.nn.dropout(x, rate=.5, noise_shape=[1, 1])
Then randomly it will give either these as weights:然后随机地给出这些作为权重:
<tf.Tensor: shape=(3, 5), dtype=float32, numpy=
array([[2., 2., 2., 2., 2.],
[2., 2., 2., 2., 2.],
[2., 2., 2., 2., 2.]], dtype=float32)>
<tf.Tensor: shape=(3, 5), dtype=float32, numpy=
array([[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0.]], dtype=float32)>
You can test it like this with a Keras layer:您可以使用 Keras 层像这样测试它:
tf.keras.layers.Dropout(rate=.5, noise_shape=[1, 1])(x, training=True)
If you use it in a model, just remove the training
argument, and make sure you manually specify the noise_shape
.如果您在 model 中使用它,只需删除training
参数,并确保手动指定noise_shape
。
Something like this should work, although I haven't tested it:像这样的东西应该可以工作,虽然我还没有测试过:
class SubclassedModel(tf.keras.Model):
def __init__(self):
super(SubclassedModel, self).__init__()
self.dense = tf.keras.layers.Dense(4)
def call(self, inputs, training=None, mask=None):
noise_shape = tf.ones(tf.rank(inputs))
x = tf.keras.layers.Dropout(rate=.5,
noise_shape=noise_shape)(inputs, training=training)
x = self.dense(x)
return x
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.