Tensorflow 2 Hub: How can I obtain the output of an intermediate layer?

Question

I am trying to implement following network Fots for Text detection using the new tensorflow 2. The authors use the resnet as the backbone of their network, so my first thought was to use the tensoflow hub resnet for loading a pretrained network. But the problem is that i can not find a way to print the summary of the module, which is loaded from tfhub?

Is there any way to see the layers of the loaded modules from tf-hub? Thanks

---

Update

Unfortunately is the resnet not available for tf2-hub, so i deceided to use the builtin keras implementation of resent, at least till there is a hub impl. of it.

Here is how i get the intermediate layers of resnet using tf2.keras.applications:

import numpy as np
import tensorflow as tf
from tensorflow import keras

layers_out = ["activation_9", "activation_21", "activation_39", "activation_48"]

imgs = np.random.randn(2, 640, 640, 3).astype(np.float32)
model = keras.applications.resnet50.ResNet50(input_shape=(640, 640, 3), include_top=False)
intermid_outputs= [model.get_layer(layer_name).output for layer_name in layers_out]
shared_conds = keras.Model(inputs=model.input, outputs=intermid_outputs)
Y = conv_shared(imgs)
shapes = [y.shape for y in Y]
print(shapes)

Answer 1

You can do something like this to examine the intermediate outputs:

resnet = hub.Module("https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/3")
outputs = resnet(np.random.rand(1,224,224,3), signature="image_feature_vector", as_dict=True)
for intermediate_output in outputs.keys():
    print(intermediate_output)

Then, if you want to link an intermediate layer of the hub module to the rest of your graph, you can do:

resnet = hub.Module("https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/3")
features = resnet(images, signature="image_feature_vector", as_dict=True)["resnet_v2_50/block4"]
flatten = tf.reshape(features, (-1, features.shape[3]))

Assuming that we want to extract the features from the last block of the ResNet.

Answer 2

Assuming that you wanted to get the intermidiate outputs, and updating @gorjan answer for newer versions of resnet in tfhub, you could try something like this.

---

First load the model using the hub.KerasLayer with the argument return_endpoints=True (this is not available for all models and is not documented afaik, results may vary):

hub_model_layer = hub.KerasLayer("https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/5",
                                     trainable=True,  
                                     arguments=dict(return_endpoints=True))

Then you can compile a model like this:

input = tf.keras.layers.Input((244, 244, 3))
dict_output = hub_model_layer(input)

C2 = dict_output['resnet_v2_50/block1/unit_1/bottleneck_v2/shortcut']
C3 = dict_output['resnet_v2_50/block2/unit_1/bottleneck_v2/shortcut']
C4 = dict_output['resnet_v2_50/block3/unit_1/bottleneck_v2/shortcut']
C5 = dict_output['resnet_v2_50/block4/unit_1/bottleneck_v2/shortcut']

model = tf.keras.Model(inputs=input, outputs=[C2, C3, C4, C5])

The variable dict_output is a dictionary with all the avilable endpoints, you can print it to search for the outputs that you want to use. They are not in order and I was not abe to find a way to recover the graph, but you can guess them by the layer names.