使用预先训练的模型训练模型

Question

I have a pre-trained model in Keras. I would like to train another model where the output of the model is input for the already trained model and the output of the already trained model is being used in the loss function of the untrained model. Something like

in_a + mod_b(some kind of feedback from B here) --> Model A --> out_a --> Model B --> out_b

error  = (in_a - out_b)**2

and then use this error to train Model A. in_a can be treated as a constant in this system and there is a feedback loop as well

Any ideas how to do this in keras or tensorflow

Answer 1

Here's an idea. Build Model A until the output layer, which we'll assume is compatible with Model B's input layer. Also, let's assume you're Using a pretrained VGG16 as Model B. You'll load the model with pretrained weights:

from keras.applications.vgg16 import VGG16

# Model A is trainable
x = Input(shape=(32,))
x_d = Dense(10)(x)
model_a_out = Dense(10)(x_d)

# Model B
model_b = VGG16(weights='imagenet', include_top=True)
# Freeze Model B
for layer in model_b.layers:
    layer.trainable = False

# Set input to Model B as output from A
model_b.input = model_a_out

# Train as usual
model_b.compile... and model_b.fit ...

Another way of doing it is first you want to build A. and then:

for layer in model_b.layers:
  new_layer = layer
  new_layer.trainable = False
  model_a.add(new_layer)

Check out the Keras Applications page for some ideas.