Saliency Map for Keras CNN LSTM (TF Backend)

Question

I am trying to get the saliency maps from my neural network, but I am struggling a little. My network does DNA Classification (similar to text classification), and is sequential as follows: Conv1D-> MaxPool-> Dropout-> Bidirectional LSTM-> Flatten-> Dense-> Dropout-> Dense. Running Keras 2.06, Tensorflow 1.2.1. Code that I got primary from https://github.com/fchollet/keras/issues/1777 is as follows:

def compile_saliency_function(model):
    """
    Compiles a function to compute the saliency maps and predicted classes
    for a given minibatch of input images.
    """
    inp = model.layers[0].input
    outp = model.layers[-1].output
    max_outp = K.max(outp, axis=1)
    saliency = keras.backend.gradients(keras.backend.sum(max_outp), inp)
    max_class = K.argmax(outp, axis=1)
    return K.function([inp], [saliency, max_class])

sal = compile_saliency_function(model)([x_test[1], 0])

but I run into the error message from K.function

TypeError: Can not convert a list into a Tensor or Operation.

What am I missing here? I would try to use other APIs but it doesn't seem like there is a whole lot out there for text, other than DeepLIFT, which doesn't support RNNs and still uses Keras 1.x. Implementation for text in Keras-vis is somewhere in the future. Any feedback is appreciated!

Answer 1

I know it is an old question, but try this: change the line

saliency = keras.backend.gradients(keras.backend.sum(max_outp), inp)

to

saliency = keras.backend.gradients(keras.backend.sum(max_outp), inp)[0]

or simply

saliency = K.gradients(keras.backend.sum(max_outp), inp)[0]