Keras merge/concatenate models outputs as a new layers

Question

I want to use pretrained models' convolutionnal feature maps as input features for a master model.

inputs = layers.Input(shape=(100, 100, 12))
sub_models = get_model_ensemble(inputs)
sub_models_outputs = [m.layers[-1] for m in sub_models]
inputs_augmented = layers.concatenate([inputs] + sub_models_outputs, axis=-1)

Here is the key part of what I do in get_model_ensemble() :

for i in range(len(models)):
    model = models[i]
    for lay in model.layers:
        lay.name = lay.name + "_" + str(i)
    # Remove the last classification layer to rather get the underlying convolutional embeddings
    model.layers.pop()
    # while "conv2d" not in model.layers[-1].name.lower():
    #     model.layers.pop()
    model.layers[0] = new_input_layer
return models

All this gives:

Traceback (most recent call last):
  File "model_ensemble.py", line 151, in <module>
    model = get_mini_ensemble_net()
  File "model_ensemble.py", line 116, in get_mini_ensemble_net
    inputs_augmented = layers.concatenate([inputs] + sub_models_outputs, axis=-1)
  File "/usr/local/lib/python3.4/dist-packages/keras/layers/merge.py", line 508, in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)
  File "/usr/local/lib/python3.4/dist-packages/keras/engine/topology.py", line 549, in __call__
    input_shapes.append(K.int_shape(x_elem))
  File "/usr/local/lib/python3.4/dist-packages/keras/backend/tensorflow_backend.py", line 451, in int_shape
    shape = x.get_shape()
AttributeError: 'BatchNormalization' object has no attribute 'get_shape'

Here is type info:

print(type(inputs))
print(type(sub_models[0]))
print(type(sub_models_outputs[0]))

<class 'tensorflow.python.framework.ops.Tensor'>
<class 'keras.engine.training.Model'>
<class 'keras.layers.normalization.BatchNormalization'>

Note: the models I get from get_model_ensemble() have got their compile() function already called. So, how should I concatenate my models properly? Why wont it work? I guess that maybe that has something to do with how would the inputs be fed to the sub-models and how I hot-swapped their input layers.

Thanks for the help!

Answer 1

The thing works if we do:

sub_models_outputs = [m(inputs) for m in sub_models]

rather than:

sub_models_outputs = [m.layers[-1] for m in sub_models]

TLDR: models needs to be called as a layer.