Add new information between convolutional layers and dense layers

Question

I want to mix result of simple classifier with the results of convolutional network to get more sophisticated classifier and test it.

Right now, I'm using keras example for Inceptionv3 net

    base_model = InceptionV3(include_top=False, weights='imagenet', 
    input_shape=(200,200,3))
    x = base_model.output
    x = GlobalAveragePooling2D()(x)
    x = Dense(1024, activation = 'relu')(x)
    predictions = Dense(NUM_CLASSES, activation='softmax')(x)
    model = Model(inputs=base_model.input, outputs=predictions)

I want to add class obtained from other classifier to the result of the first Dense layer, but don't understand how to send it to model.

Thanks.

Answer 1

You need a model with two branches.

input1 = Input(yourInputShape)

Follow the same line until the dense you want to change the results:

v3out = base_model(input1)
v3out =  GlobalAveragePooling2D()(v3out)
v3out = Dense(1024, activation = 'relu')(v3out)

Have a similar construction for the other classifier:

otherOut = other_model(input1)
otherOut = MoreLayers(....)(otherOut)

Now you can either sum them if they have the same shape, or concatenate them (append one to the end of the other)

#sum
joined = Add()([v3Out,otherOut])

#or concatenate
joined = Concatenate()([v3Out,otherOut])

Finish the model as normally:

predictions = Dense(NUM_CLASSES, activation='softmax')(joined)
model = Model(inputs=input, outputs=predictions)