How can I obtain the output of an intermediate layer?

Question

I'm trying to understand Google's colab code . How should I use this code:

from keras import backend as K
prediction_model = lstm_model(seq_len=1, batch_size=BATCH_SIZE, stateful=True)
prediction_model.load_weights('/tmp/bard.h5')

get_test_layer_output = K.function([prediction_model.layers[0].input],
                                  [prediction_model.layers[1].output])
layer_output = get_test_layer_output([x])[0]

to see the values after each layer? Or is there any different approach to see the values (not shapes)?

Layer (type)                 Output Shape              Param #   
=================================================================
seed (InputLayer)            (128, 100)                0         
_________________________________________________________________
embedding (Embedding)        (128, 100, 512)           131072    
_________________________________________________________________
lstm (LSTM)                  (128, 100, 512)           2099200   
_________________________________________________________________
lstm_1 (LSTM)                (128, 100, 512)           2099200   
_________________________________________________________________
time_distributed (TimeDistri (128, 100, 256)           131328    
=================================================================
Total params: 4,460,800
Trainable params: 4,460,800
Non-trainable params: 0

Answer 1

For any operation which is to be carried out on the layers of a Keras model, first, we need to access the list of keras.layers object which a model holds.

model_layers = model.layers

Each Layer object in this list has its own input and output tensors ( if you're using the TensorFlow backend )

input_tensor = model.layers[ layer_index ].input
output_tensor = model.layers[ layer_index ].output

If you directly run the output_tensor with the tf.Session.run() method you will get an error stating inputs must be fed to the model before accessing the outputs of the layers .

import tensorflow as tf
import numpy as np

layer_index = 3 # The index of the layer whose output needs to be fetched

model = tf.keras.models.load_model( 'model.h5' )
out_ten = model.layers[ layer_index ].output

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    output = sess.run(  out_ten , { model.input : np.ones((2,186))}  ) 
    print( output )

You need to initialize the variables using tf.global_variables_initializer().run() before running the model. The model.input provides the placeholder tensor for the model's input.