Tensorflow：将手动构建图层转换为tf.contrib.layers

Question

I have these four layers defined:

layer_1 = tf.add(
    tf.matmul(input, tf.Variable(tf.random_normal([n_input, n_hidden_1])),
    tf.Variable(tf.random_normal([n_hidden_1]))))
layer_2 = tf.nn.sigmoid(tf.add(
    tf.matmul(layer_1, tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    tf.Variable(tf.random_normal([n_hidden_2]))))
layer_3 = tf.nn.sigmoid(tf.add(
    tf.matmul(layer_2, tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])))),
    tf.Variable(tf.random_normal([n_hidden_1]))))
layer_4 = tf.add(
    tf.matmul(layer_3, tf.Variable(tf.random_normal([n_hidden_1, n_input]))),
    tf.Variable(tf.random_normal([n_input])))

I would like to transform this code into code based on tf.contrib.layers . So far I got

layer_1 = tf.contrib.layers.fully_connected(
    inputs=input,
    num_outputs=n_hidden_1,
    activation_fn=None)
layer_2 = tf.contrib.layers.fully_connected(
    inputs=layer_1,
    num_outputs=n_hidden_2,
    activation_fn=tf.nn.sigmoid)
layer_3 = tf.contrib.layers.fully_connected(
    inputs=layer_2,
    num_outputs=n_hidden_1,
    activation_fn=tf.nn.sigmoid)
layer_4 = tf.contrib.layers.linear(
    inputs=layer_3,
    num_outputs=n_input)

by reading up on https://www.tensorflow.org/versions/master/tutorials/layers/ and https://www.tensorflow.org/api_docs/python/tf/contrib/layers/fully_connected . I read in https://www.tensorflow.org/api_guides/python/contrib.layers#Higher_level_ops_for_building_neural_network_layers that tf.contrib.layers.linear is an alternative for the linear layer.

But my output is more different compared to what I got earlier, then that this could be by chance. What did I do wrong in the configuration of the layers?

Answer 1

One difference between your code and the tf.contrib.layers version is that the default initializers are different:

• The initializer for the weights defaults to tf.contrib.layers.xavier_initializer() .
• The initializer for the biases defaults to tf.zeros_initializer() .

These are generally considered to be good defaults for a fully connected layer, but you can override them with a tf.random_normal_initializer as follows:

layer_1 = tf.contrib.layers.fully_connected(
    inputs=input,
    num_outputs=n_hidden_1,
    activation_fn=None,
    weights_initializer=tf.random_normal_initializer(),
    biases_initializer=tf.random_normal_initializer())
# ...