Keras：输出wrt输入的梯度作为分类器的输入

Question

I am doing research and for an experiment I want to use gradients of a specific layer in the network with respect to the network's input( similar as guided backprop) as input to another network (classifier). The goal is to 'force' network to change 'attention' according to classifier, so those two networks should be trained simultaneously.

I implemented it on this way :

input_tensor = model.input
output_tensor = model.layers[-2].output
grad_calc = keras.layers.Lambda(lambda x:K.gradients(x,input_tensor)[0],output_shape=(256,256,3),trainable=False)(output_tensor)
pred = classifier(grad_calc)
out_model = Model(input_tensor,pred)

out_model.compile(loss='mse',optimizer=keras.optimizers.Adam(0.0001),metrics=['accuracy'])

Then, when I try to train the model

out_model.train_on_batch(imgs,np.zeros((imgs.shape[0],2)))

it is not working. It seems that it stucks there, nothing is happening (no error nor other message).

I am not sure is this right way to implement this, so I would be very thankful if someone with more experience can take a look and give me advice.

Answer 1

If I was trying to achieve that I would swith to plain Tensorflow and something along the lines:

#build model
input = tf.placeholder()
net   = tf.layesr.conv2d(input, 12)
loss  = tf.nn.l2_loss(net)
step  = tf.train.AdamOptimizer().minimize(loss)

# now inspect your graph and select the gradient tensor you are looking for
for op in tf.get_default_graph.get_operations():
    print(op.name)
grad = tf.get_default_graph().get_operation_by_name("enqueue")

with tf.Session as sess:
    _, grad, input = sess.run([step, grad, input], ...)
    # feed your grad and input into another network