How to put tf.layers variables in tf.name_scope/tf.variable_scope?

Question

I have a problem with Tensorflow:

The following code produces a correct(ish) graph for a convolutional block:

def conv_layer(self, inputs, filter_size = 3, num_filters = 256, name = None):
    scope_name = name
    if name == None:
        scope_name = "conv_layer"

    with tf.name_scope(scope_name):
        conv = tf.contrib.layers.conv2d(inputs, num_filters, filter_size, activation_fn = None)
        batch_norm = tf.contrib.layers.batch_norm(conv)
        act = tf.nn.leaky_relu(batch_norm)

        return act

The problem is that the tf.layers API makes some ugly variables that do not actually stay within the name_scope . Here is the Tensorboard view so you can see what I mean.

Is there anyway to get those variables to go into the scope? This is a big problem when it comes to visualizing the graph because I plan this network to much larger. (As you can see to the right, this is already a big problem, I have to remove those from the main graph manually every time I boot up Tensorboard.)

Answer 1

You can try using tf.variable_scope instead. tf.name_scope is ignored by variables created via tf.get_variable() which is usually used by tf.layers functions. This is in contrast to variables created via tf.Variable .

See this question for an (albeit somewhat outdated) explanation of the differences.

Answer 2

Solution moved from question to an answer:

Changing each instance of name_scope with variable_scope the problem has been omitted. However, I had to assign each variable_scope with a unique ID and set reuse = False .

def conv_layer(self, inputs, filter_size = 3, num_filters = 256, name = None):
    scope_name = name
    if name == None:
        scope_name = "conv_layer_" + str(self.conv_id)
        self.conv_id += 1

    with tf.variable_scope(scope_name, reuse = False):
        conv = tf.contrib.layers.conv2d(inputs, num_filters, filter_size, activation_fn = None)
        batch_norm = tf.contrib.layers.batch_norm(conv)
        act = tf.nn.leaky_relu(batch_norm)

        return act

As you can see, the variables are nicely hidden away in the correct blocks.