GradientTape 中的多个训练变量 tensorflow

Question

I have the following list of trainable_variables in tensorflow:

l_vars = [l_unary.trainable_variables, l_regularization.trainable_variables]

which is passed into the tape like so:

    grad_l_model = l_tape.gradient(tf.constant(l_loss), l_vars)

However, this gives the error:

AttributeError: 'list' object has no attribute '_in_graph_mode'

when wrapping l_vars with tf.Variable, I get the error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Shapes of all inputs must match: values[0].shape = [5,5,3,32] != values[1].shape = [32] [Op:Pack] name: initial_value

tf.keras.layers.concatenate([l_unary.trainable_variables, l_regularization.trainable_variables], 0)

gives:

ValueError: A Concatenate layer should be called on a list of at least 2 inputs

How do I train multiple model variables with tensorflow together?

Answer 1

Note that your l_vars is not a list, but a list of lists, since each trainable_variables attribute is already a list in itself. You can simply use

l_vars = l_unary.trainable_variables + l_regularization.trainable_variables

to concatenate the two lists into one big list.