How will I get prediction from this ensembled model?

Question

# Load the hdf5 files
from keras.models import load_model

resnet50 = keras.models.load_model('/content/drive/MyDrive/HDF5 Files/RestNet 50 best_model.hdf5')
resnet152 = keras.models.load_model('/content/drive/MyDrive/HDF5 Files/best_model_4.hdf5')

# Get the predictions from each model
predictions1 = resnet50.predict(images)
predictions3 = resnet152.predict(images)

# Combine the predictions using a majority vote
predictions = np.array([predictions1,  predictions3])
predictions = np.mean(predictions, axis=0)
print(predictions)

. The output I'm getting is [[9.9993783e-01 1.3912816e-06 6.0800008e-05 2.9077312e-09]]. What does this mean?

Answer 1

Its not clear from your question how many images you are passing and how many categories for classification you have in your problem. I will try to answer in a generic sense.

Say you are passing a batch of 4 images to each of the model restnet50 and resnet152 and both of them are trained on 5 categories then each model will give prediction with shape (4,5).

>> predictions1.shape
(4,5)
>> predictions3.shape
(4,5)

To combine them for majority vote, you should code as follows:

>>predictions = np.dstack((predictions1, predictions3)) # stack along the third axis.
>>predictions.shape
(4,5,2)
>> mean_predictions = np.mean(predictions, axis=2) # find the mean probabilities of both predictions in each category.
>> mean_predictions.shape
(4,5)
>> class_index = np.argmax(mean_predictions,axis=1) # Find the index having highest probability.
>> class_index.shape
(4,)

The final class_index gives the index of class or category to which each of the 4 images belong. I hope this helps.

BETTER WAY You can create ensemble model in tensorflow in a much better way as follows:

from tensorflow.keras.applications.resnet50 import preprocess_input as process_resnet50
from tensorflow.keras.applications.resnet_v2 import preprocess_input as process_resnetv2
from tensorflow.keras.layers import Lambda,Dense,GlobalMaxPool2D,Concatenate
from tensorflow.keras import Input, Model
from keras.models import load_model

resnet50 = keras.models.load_model('/content/drive/MyDrive/HDF5 Files/RestNet 50 best_model.hdf5')
resnet152 = keras.models.load_model('/content/drive/MyDrive/HDF5 Files/best_model_4.hdf5')


category_count = 5 # you can give your own category count.
inp = Input(input_shape)
resnet_prep50 = tf.keras.layers.Lambda(process_resnet50)(inp)
res_net50 = resnet50(resnet_prep50)
x_resnet50 = GlobalMaxPool2D()(res_net50)
x_resnet50 = Dense(128, activation='relu')(x_resnet50)

resnet_prep152 = tf.keras.layers.Lambda(process_resnetv2)(inp)
res_net152 = resnet152(resnet_prep152)
x_resnet152 = GlobalMaxPool2D()(res_net152)
x_resnet152 = Dense(128, activation='relu')(x_resnet152)

x = Concatenate()([x_resnet50, x_resnet152])
out = Dense(category_count, activation='softmax')(x)

ensemble_model = Model(inputs=inp, outputs = out)
predictions = ensemble_model.predict(images)
predictions = np.argmax(predictions, axis=1)