如何將 Conv2d 層 output 作為 Keras model 的輸入？

Question

如何在 Keras model 之上添加一層 Conv2D？ 我的輸入形狀為（299,299,15），為了使用預訓練權重（imagenet），輸入通道必須為 3，因此我的想法是添加一個 conv2d 層，將通道從 15 更改為 3。

image = Input(shape=(299, 299, 15))
x = Conv2D(3, kernel_size=(8,8), strides=(2,2), activation='relu')(image)
model1 = Model(inputs=image, outputs=x)

model2 = InceptionResNetV2(include_top=False, weights = 'imagenet', input_tensor=None, input_shape=(299,299,3))

Answer 1

嘗試

image = Input(shape=(299, 299, 15))
x = Conv2D(3, kernel_size=(8,8), strides=(2,2), activation='relu')(image)
model1 = Model(inputs=image, outputs=x)
x=model1.output
x=tf.keras.applications.InceptionResNetV2(include_top=False, weights = 'imagenet', input_tensor=None)(x)
model2=Model(inputs=image, outputs=x)
print(model2.summary())

您可能希望將 pooling='max' 參數添加到 InceptionResNetV2 參數。 這將導致 output 成為您可以輸入密集層的一維向量。 model 總結是

Model: "functional_3"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 299, 299, 15)]    0         
_________________________________________________________________
conv2d (Conv2D)              (None, 146, 146, 3)       2883      
_________________________________________________________________
inception_resnet_v2 (Functio (None, None, None, 1536)  54336736  
=================================================================
Total params: 54,339,619
Trainable params: 54,279,075
Non-trainable params: 60,544

Answer 2

This will first create a model that takes a x_input=(229,229,15) as input and performs convolution to reduce the channels to 3. The output of this model is then fed to the base_ model (InceptionResNetV2) and a few layers are added such作為GlobalAveragePooling和Dense層。 最終的 model 是使用x_input作為第一層和Dense層預測 10 個類作為 output 層構建的。

import tensorflow as tf
from tensorflow.keras.models import Model
from tensorflow.keras.layers import Dense, GlobalAveragePooling2D

# define input
x_input = tf.keras.layers.Input(shape=(229, 229, 15))
# convolve to go from 15 channels to 3
x_conv = tf.keras.layers.Conv2D(3,1)(x_input)
# model that performs convolution
conv_model = Model(inputs=x_input, outputs=x_conv)
# storing the model output, which will be later used as input for the base model
conv_output=conv_model.output

# defining base model
base_model = tf.keras.applications.InceptionResNetV2(
    weights='imagenet',
    include_top=False
)(conv_output)

# add a global spatial average pooling layer
x = GlobalAveragePooling2D()(base_model)
# let's add a fully-connected layer
x = Dense(1024, activation='relu')(x)
# and a logistic layer -- let's say we have 10 classes
predictions = Dense(10, activation='softmax')(x)

# this is the model we will train
model = Model(inputs=x_input, outputs=predictions)

model.summary()

查看 Model 總結