如何在层次模型中获得注意力权重

Question

Model :

sequence_input = Input(shape=(MAX_SENT_LENGTH,), dtype='int32')
words = embedding_layer(sequence_input)
h_words = Bidirectional(GRU(200, return_sequences=True,dropout=0.2,recurrent_dropout=0.2))(words)
sentence = Attention()(h_words)  #with return true
#sentence = Dropout(0.2)(sentence)
sent_encoder = Model(sequence_input, sentence[0])
print(sent_encoder.summary())

document_input = Input(shape=(None, MAX_SENT_LENGTH), dtype='int32')
document_enc = TimeDistributed(sent_encoder)(document_input)
h_sentences = Bidirectional(GRU(100, return_sequences=True))(document_enc)

preds = Dense(7, activation='softmax')(h_sentences)
model = Model(document_input, preds)

Attention layer used: https://gist.github.com/cbaziotis/6428df359af27d58078ca5ed9792bd6d with return_attention=True

How can I visualise attention weights for a new input once the model is trained.

What I am trying:

 get_3rd_layer_output = K.function([model.layers[0].input,K.learning_phase()],
[model.layers[1].layer.layers[3].output])

and passing a new input but it is giving me error.

Possible reasons: model.layers() only gives me last layers. I want ot get weights from the Timedistributed part.

Answer 1

You can use the following to display all the layers in your model:

print(model.layers)

Once you know what index number is your Time Distributed layer, say, 3, then use the following to get the config and the layer weights.

g = model_name.layers[3].get_config()
h = model_name.layers[3].get_weights()
print(g)
print(h)