[英]Custom weight initialization tensorflow tf.layers.dense
I'm trying to set up custom initializer to tf.layers.dense
where I initialize kernel_initializer
with a weight matrix I already have.我正在尝试为tf.layers.dense
设置自定义初始化器,我在其中使用已有的权重矩阵初始化kernel_initializer
。
u_1 = tf.placeholder(tf.float32, [784, 784])
first_layer_u = tf.layers.dense(X_, n_params, activation=None,
kernel_initializer=u_1,
bias_initializer=tf.keras.initializers.he_normal())
This is throwing error saying ValueError: If initializer is a constant, do not specify shape.
这是抛出错误,说ValueError: If initializer is a constant, do not specify shape.
Is it a problem to assign placeholder to kernel_initializer
or am I missing something?将占位符分配给kernel_initializer
是一个问题还是我遗漏了什么?
There are at least two ways to achieve this:至少有两种方法可以实现这一点:
1 Create your own layer 1 创建自己的图层
W1 = tf.Variable(YOUR_WEIGHT_MATRIX, name='Weights')
b1 = tf.Variable(tf.zeros([YOUR_LAYER_SIZE]), name='Biases') #or pass your own
h1 = tf.add(tf.matmul(X, W1), b1)
2 Use the tf.constant_initializer
2 使用tf.constant_initializer
init = tf.constant_initializer(YOUR_WEIGHT_MATRIX)
l1 = tf.layers.dense(X, o, kernel_initializer=init)
I think you can define your own initializer function.我认为您可以定义自己的初始化函数。 The function needs to take 3 arguments: shape
, dtype
, and partition_info
.该函数需要采用 3 个参数: shape
、 dtype
和partition_info
。 It should return a tf.Tensor
which will be used to initialize the weight.它应该返回一个用于初始化权重的tf.Tensor
。 Since you have a numpy
array, I think you can use tf.constant
to create this tensor.由于你有一个numpy
数组,我认为你可以使用tf.constant
来创建这个张量。 For example:例如:
def custom_initializer(shape_list, dtype, partition_info):
# Use np.ones((7, 3)) as an example
return tf.constant(np.ones((7, 3)))
Then you can pass it to kernel_initializer
.然后你可以将它传递给kernel_initializer
。 It should work if dimensions all match.如果尺寸都匹配,它应该可以工作。 I put an example on gist using Estimator
to construct the model and using LoggingTensorHook
to record dense/kernel
at each step.我在gist上举了一个例子,使用Estimator
构建模型并使用LoggingTensorHook
在每一步记录dense/kernel
。 You should be able to see that the weight is initiated correctly.您应该能够看到重量已正确启动。
I just found that using tf.constant_initializer
will be better.我刚刚发现使用tf.constant_initializer
会更好。 It is used in tensorflow guide .它用于tensorflow 指南。 You can do kernel_initializer=tf.constant_initializer(np.ones((7, 3)))
.你可以做kernel_initializer=tf.constant_initializer(np.ones((7, 3)))
。
Jonathan's answer worked for me on conv as well -乔纳森的回答也对我有用 conv -
kernel_in = np.random.uniform(100,1000,(filter_width, filter_height, input_channels, output_channels)).astype(np.float32)
init = tf.constant_initializer(kernel_in)
def model(x):
x = tf.layers.conv2d(x, filters=3, kernel_size=1, strides=1, kernel_initializer=init)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.