Tensorflow中的可变范围问题

Question

def biLSTM(data, n_steps):


    n_hidden= 24
    data = tf.transpose(data, [1, 0, 2])
    # Reshape to (n_steps*batch_size, n_input)
    data = tf.reshape(data, [-1, 300])
    # Split to get a list of 'n_steps' tensors of shape (batch_size, n_input)
    data = tf.split(0, n_steps, data)    

    lstm_fw_cell = tf.nn.rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
    # Backward direction cell
    lstm_bw_cell = tf.nn.rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)

    outputs, _, _ = tf.nn.bidirectional_rnn(lstm_fw_cell, lstm_bw_cell, data, dtype=tf.float32)


    return outputs, n_hidden

In my code I am calling this function twice to create 2 bidirectional LSTMs. Then I got the problem of reusing variables.

ValueError: Variable lstm/BiRNN_FW/BasicLSTMCell/Linear/Matrix already exists, disallowed. Did you mean to set reuse=True in VarScope?

To resolve this I added the LSTM definition in the function within with tf.variable_scope('lstm', reuse=True) as scope:

This led to a new issue

ValueError: Variable lstm/BiRNN_FW/BasicLSTMCell/Linear/Matrix does not exist, disallowed. Did you mean to set reuse=None in VarScope?

Please help with a solution to this.

Answer 1

When you create BasicLSTMCell(), it creates all the required weights and biases to implement an LSTM cell under the hood. All of these variables are assigned names automatically. If you call the function more than once within the same scope you get the error you get. Since your question seems to state that you want to create two separate LSTM cells, you do not want to reuse the variables, but you do want to create them in separate scopes. You can do this in two different ways (I haven't actually tried to run this code, but it should work). You can call your function from within a unique scope

def biLSTM(data, n_steps):
    ... blah ...

with tf.variable_scope('LSTM1'):
    outputs, hidden = biLSTM(data, steps)

with tf.variable_scope('LSTM2'):
    outputs, hidden = biLSTM(data, steps)

or you can pass a unique scope name to the function and use the scope inside

def biLSTM(data, n_steps, layer_name):
    ... blah...
    with tf.variable_scope(layer_name) as scope:
        lstm_fw_cell = tf.nn.rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
        lstm_bw_cell = tf.nn.rnn_cell.BasicLSTMCell(n_hidden, forget_bias=1.0)
        outputs, _, _ = tf.nn.bidirectional_rnn(lstm_fw_cell, lstm_bw_cell, data, dtype=tf.float32)
    return outputs, n_hidden

l1 = biLSTM(data, steps, 'layer1')
l2 = biLSTM(data, steps, 'layer2')

It is up to your coding sensibilities which approach to choose, they are functionally pretty much the same.

Answer 2

I also has the similar problem. However I was using keras implementation with pretrained Resnet50 model.

It worked for me when I updated the tensorflow version using following command:

conda update -f -c conda-forge tensorflow

and used

from keras import backend as K
K.clear_session