在张量流中计算新状态（RNN）相对于模型参数（包括用于输入的CNN）的梯度； tf.gradient返回无

Question

Summary:

I have a 1d CNN that extracts features from the inputs. The CNN is then follows by an RNN. I am looking for a way to back propagate gradients from the new_state of the RNN to the CNN parameters. Also, we can consider a conv layer with kernel size [1, 1, input_num_features, output_num_features] . The code is below:

import tensorflow as tf

mseed = 123
tf.set_random_seed(mseed)
kernel_initializer = tf.glorot_normal_initializer(seed=mseed)

# Graph Hyperparameters
cell_size = 64
num_classes = 2
m_dtype = tf.float32
num_features = 30

inputs_train_ph = tf.placeholder(dtype=m_dtype, shape=[None, 75, num_features], name="inputs_train_ph")
inputs_devel_ph = tf.placeholder(dtype=m_dtype, shape=[None, 75, num_features], name="inputs_devel_ph")

labels_train_ph = tf.placeholder(dtype=m_dtype, shape=[None, 75, num_classes], name="labels_train_ph")
labels_devel_ph = tf.placeholder(dtype=m_dtype, shape=[None, 75, num_classes], name="labels_devel_ph")

def return_inputs_train(): return inputs_train_ph
def return_inputs_devel(): return inputs_devel_ph
def return_labels_train(): return labels_train_ph
def return_labels_devel(): return labels_devel_ph

phase_train = tf.placeholder(tf.bool, shape=())
dropout = tf.placeholder(dtype=m_dtype, shape=())
initial_state = tf.placeholder(shape=[None, cell_size], dtype=m_dtype, name="initial_state")

inputs = tf.cond(phase_train, return_inputs_train, return_inputs_devel)
labels = tf.cond(phase_train, return_labels_train, return_labels_devel)

# Graph
def model(inputs):
    used = tf.sign(tf.reduce_max(tf.abs(inputs), 2))

    length = tf.reduce_sum(used, 1)
    length = tf.cast(length, tf.int32)

    with tf.variable_scope('layer_cell'):
        inputs = tf.layers.conv1d(inputs, filters=100, kernel_size=3, padding="same",
                                  kernel_initializer=tf.glorot_normal_initializer(seed=mseed))
        inputs = tf.layers.batch_normalization(inputs, training=phase_train, name="bn")
        inputs = tf.nn.relu(inputs)

    with tf.variable_scope('lstm_model'):

        cell = tf.nn.rnn_cell.GRUCell(cell_size, kernel_initializer=kernel_initializer)
        cell = tf.nn.rnn_cell.DropoutWrapper(cell, input_keep_prob=1.0 - dropout, state_keep_prob=1.0 - dropout)
        output, new_state = tf.nn.dynamic_rnn(cell, inputs, dtype=m_dtype, sequence_length=length,
                                              initial_state=initial_state)

    with tf.variable_scope("output"):
        output = tf.reshape(output, shape=[-1, cell_size])
        output = tf.layers.dense(output, units=num_classes,
                                 kernel_initializer=kernel_initializer)

        output = tf.reshape(output, shape=[5, -1, num_classes])
        used = tf.expand_dims(used, 2)

        output = output * used

    return output, new_state


output, new_state = model(inputs)

grads_new_state_wrt_vars = tf.gradients(new_state, tf.trainable_variables())
for g in grads_new_state_wrt_vars:
    print('**', g)

init_op = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init_op)

Please note that when I printed out the gradients tensors, I got the following:

for g in grads_new_state_wrt_vars:
    print('**', g)

** None
** None
** None
** None
** Tensor("gradients/model/lstm_model/rnn/while/gru_cell/MatMul/Enter_grad/b_acc_3:0", shape=(220, 240), dtype=float64)
** Tensor("gradients/model/lstm_model/rnn/while/gru_cell/BiasAdd/Enter_grad/b_acc_3:0", shape=(240,), dtype=float64)
** Tensor("gradients/model/lstm_model/rnn/while/gru_cell/MatMul_1/Enter_grad/b_acc_3:0", shape=(220, 120), dtype=float64)
** Tensor("gradients/model/lstm_model/rnn/while/gru_cell/BiasAdd_1/Enter_grad/b_acc_3:0", shape=(120,), dtype=float64)
** None
** None

Finally, the weights in the network are printed below:

for v in tf.trainable_variables():
    print(v.name)

model/conv1d/kernel:0
model/conv1d/bias:0
model/bn/gamma:0
model/bn/beta:0
model/lstm_model/rnn/gru_cell/gates/kernel:0
model/lstm_model/rnn/gru_cell/gates/bias:0
model/lstm_model/rnn/gru_cell/candidate/kernel:0
model/lstm_model/rnn/gru_cell/candidate/bias:0
model/output/dense/kernel:0
model/output/dense/bias:0

Therefore, how come that the gradients can't be computed wrt to the weights of the first conv, and batch norm, layers in the network?

Please note that I don't have the same problem when replacing new_state by output in tf.gradients(new_state, tf.trainable_variables())

Any help is much appreciated!!

Edit

I found out that if I change the None in the placeholders defined above, the problem will be solved. And I got the gradients of the new_state wrt to the conv layers. This will work as long as the defined batch size is the same in both train and devel placeholders , ex:

inputs_train_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_features], name="inputs_train_ph")
inputs_devel_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_features], name="inputs_devel_ph")

labels_train_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_classes], name="labels_train_ph")
labels_devel_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_classes], name="labels_devel_ph")

Else, I will run into an error again.

Please note that the gradients of the output wrt conv layer won't be affected by the None for the batch size in the placeholders defined above.

Now I would like to know why I am getting this error had I didn't change the None to batch_size ?

Answer 1

I'm not sure whether this constitutes an exact answer to your question, but it might be a good start as it seems your issue does not get much of an attention (and well, tbh, I'm not surprised with this amount of confusion).

Furthermore there are tons of questions (way too much for comments) others might have as well so I will bring them up here (and try to provide an answer to the situation at hand).

Tensorflow version: 1.12

1. None gradients

This problem does not exist for either output or new_state when batch_size is unspecified .

• Gradients wrt new_state , eg grads_new_state_wrt_vars = tf.gradients(new_state, tf.trainable_variables()) return:

   ** Tensor("gradients/layer_cell/conv1d/BiasAdd_grad/BiasAddGrad:0", shape=(100,), dtype=float32) ** Tensor("gradients/layer_cell/bn/batchnorm/mul_grad/Mul_1:0", shape=(100,), dtype=float32) ** Tensor("gradients/layer_cell/bn/batchnorm/add_1_grad/Reshape_1:0", shape=(100,), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/MatMul/Enter_grad/b_acc_3:0", shape=(164, 128), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/BiasAdd/Enter_grad/b_acc_3:0", shape=(128,), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/MatMul_1/Enter_grad/b_acc_3:0", shape=(164, 64), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/BiasAdd_1/Enter_grad/b_acc_3:0", shape=(64,), dtype=float32) ** None ** None 

  as expected (as it doesn't pass through the Dense part of your network).

• Gradients wrt output_state , eg grads_new_state_wrt_vars = tf.gradients(output_state, tf.trainable_variables()) return:

   ** Tensor("gradients/layer_cell/conv1d/BiasAdd_grad/BiasAddGrad:0", shape=(100,), dtype=float32) ** Tensor("gradients/layer_cell/bn/batchnorm/mul_grad/Mul_1:0", shape=(100,), dtype=float32) ** Tensor("gradients/layer_cell/bn/batchnorm/add_1_grad/Reshape_1:0", shape=(100,), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/MatMul/Enter_grad/b_acc_3:0", shape=(164, 128), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/BiasAdd/Enter_grad/b_acc_3:0", shape=(128,), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/MatMul_1/Enter_grad/b_acc_3:0", shape=(164, 64), dtype=float32) ** Tensor("gradients/lstm_model/rnn/while/gru_cell/BiasAdd_1/Enter_grad/b_acc_3:0", shape=(64,), dtype=float32) ** Tensor("gradients/output/dense/MatMul_grad/MatMul_1:0", shape=(64, 2), dtype=float32) ** Tensor("gradients/output/dense/BiasAdd_grad/BiasAddGrad:0", shape=(2,), dtype=float32) 

  Once again, everything is fine.

2. Specifying batch_size

When batch size is specified as you described, eg

inputs_train_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_features], name="inputs_train_ph")
inputs_devel_ph = tf.placeholder(dtype=m_dtype, shape=[34, 75, num_features], name="inputs_devel_ph")

your network DOES NOT work at all and it's nothing strange as your shapes do not match in the output namespace.

Furthermore, labels_devel_ph and labels_train_ph does not matter for the code in question, why put them here, they just clutter the question even further. Please see Minimal, Complete and Verifiable Example , there is a lot of parts totally not needed for the task in question.

Next, there is no connection between batch shape of inputs_train_ph and inputs_devel_ph , why would there be? One is independent of the other and only one is used at a time due to tf.cond (which has to be provided as a value in session run, but that's beside the scope of this question).

Problematic output part, exactly this:

output = tf.reshape(output, shape=[5, -1, num_classes])
used = tf.expand_dims(used, 2)

output = output * used

Tensor shapes using your approach:

Output shape: (5, 480, 2)                                                                                                                          
Used initial shape: (32, 75) 
Used after expand_dims: (32, 75, 1)

Obviously (5, 480, 2) multiplied by (32, 75, 1) will not work, I see no possibility of this working ever, even with Tensorflow in pre-alpha version, which makes me think there are other parts of your source code making it working and who knows what else influences it to be honest.

Problem with used can be solved in multiple ways, but I think what you wanted is to stack used in another dimension and reshape afterwards ( it will not work without reshaping ):

output = tf.reshape(output, shape=[5, -1, num_classes])
used = tf.stack((used, used), 2)
used = tf.reshape(used, shape=(5, -1, num_classes))

output = output * used

Using this approach, every batch shape passes through network without problem.

BTW. I'm not entirely sure what you are trying to achieve with used at the beginning either, but maybe it has your use case, IMO the intent is totally unreadable when compared to final result (columns containing zeros when all features in sequence are zero and one otherwise).

3. Tensorflow versions

Tested with version 1.8 of Tensorflow (current is 1.12 and 2.0 is around the corner) and I was able to reproduce your problem.

Still, output shapes crash as described above though.

Actually, version 1.9 already has this problem solved.

Why this problem exists in 1.8?

I have tried to pin-point possible reason, but I still don't know. Some ideas:

• Using tf.layers instead of tf.nn . As planned for version 2.0 this module will be deprecated and due to a lot of mess in the framework, I assumed something might be off in this case as well. I have changed conv1d to it's tf.keras.layers counterpart and batch normalization to tf.nn.batch_normalization , to no avail unluckily, results are still the same.

• According to 1.9 release notes Prevent tf.gradients() from backpropagating through integer tensors . Maybe this has something to do with your problem? Maybe backpropagation graph is somehow stuck in v1.8.0?

• Problems with scope tf.variable_scope and tf.layers - as expected, nothing changed after removing namespaces.

All in all: update your dependencies to version 1.9 or above, it solves all your problems (though why they occur are rather hard to grasp, as is this framework).

Actually it's above my head why such changes are between minor versions and not described more in-depth in the changelog, but maybe I'm missing something big...

Little off-topic

Maybe you should consider using tf.Estimator , tf.keras or another framework like PyTorch? This code is highly unreadable, hard to debug and ugly, newer practices exist and they should help you and us (if you have another problem with your neural networks and decide to ask on StackOverflow)