Tensorflow tf.layers, tf.contrib.layers not working with variable scope

Question

I am beginning to use TensorFlow for some simple Q-learning, but have run into trouble when trying to use variable scopes with layers constructed using tf.layers and tf.contrib.layers . In a nutshell, I want to apply the same layers to different input tensors (for example, to hold the current and next Q values). Here is a minimal example using tf.layers :

import tensorflow as tf

inp1 = tf.placeholder(tf.float64, (4,1))
inp2 = tf.placeholder(tf.float64, (4,1))

def process(inp):
    with tf.variable_scope("foo", reuse=True):
        return tf.layers.dense(inp, 12, name="bar", reuse=True)

process(inp1)
process(inp2)

Trying to execute this code gives the following exception:

ValueError: Variable foo/bar/kernel does not exist, or was not created with
tf.get_variable(). Did you mean to set reuse=None in VarScope?

I understand that setting reuse=True in tf.layers.dense() makes it try to find an already defined layer, which it may fail to do. But if I change the call into tf.layers.dense(inp, 12, name="bar") , then it fails with the same exception.

If I set reuse=None in tf.variable_scope() , then the latter version fails during the call of process(inp2) with the exception:

ValueError: Variable foo/bar/kernel already exists, disallowed. 
Did you mean to set reuse=True in VarScope?

Unfortunately, similar errors occur when using tf.contrib.layers .

My question is: Is there a way to make tf.layers work with variable scopes? I know that I could define the weights and biases separately, but it would be nice to retain the abstraction given by tf.layers . Thanks a lot!

My setup is TensorFlow 1.3.0 (CPU) running with Python 3.6.1 on Windows 10 (installed through pip on 64-bit Anaconda 4.4.0).

PS I found the use of variable scopes for layers on page 17 of this presentation .

Answer 1

Two errors are different: the first one happened in process(inp1) , where it tries to find the existed variables but there is not; the second happened in process(inp2) , where the variable with same name existed but it tries to create a new variable with the same name, disallowed.

I guess that you want to reuse those variables for Q-learning. So the solution is quite simple: the first time you define those variables, don't use reuse , then you can set reuse=True .

In the presentation you gave, I guess they have already defined variables before.

This guide will help you under more.