使用 ImageDataGenerator 和 flow_from_directory 時如何使用來自 TensorHub 的遷移學習和自定義圖像大小

Question

我正在嘗試學習如何從預訓練模型中執行特征提取以完成遷移學習任務。 我目前正在嘗試使用來自 tensorhub 的 MobileNet v2 特征提取器，雖然原始圖像形狀是 (224, 224) 的元組並且我的圖像是 384x288x3。 我嘗試做的是：

import tensorflow as tf
import tensorflow_hub as hub
from tensorflow.keras import layers


IMG_SHAPE = (384, 288)
BATCH_SIZE = 32

train_dir = '/content/drive/MyDrive/dataset_split/Training'
test_dir = '/content/drive/MyDrive/dataset_split/Test'


train_datagen = ImageDataGenerator(rescale=1/255.)
test_datagen = ImageDataGenerator(rescale=1/255.)


training_dataset = train_datagen.flow_from_directory(train_dir, target_size=IMG_SHAPE,
                                                     batch_size=BATCH_SIZE, class_mode='categorical')


print("Testing Images: ")
test_data = test_datagen.flow_from_directory(test_dir, target_size=IMG_SHAPE,
                                             batch_size=BATCH_SIZE, class_mode='categorical')
    
mobilenet_url = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4"



def create_model(model_url, num_classes=2):
  feature_extractor_layer = hub.KerasLayer(model_url, trainable=False, name="feature_extractor_layer", input_shape=IMG_SHAPE)
  model = tf.keras.Sequential([feature_extractor_layer, layers.Dense(num_classes, activation="softmax", name="output_layer")])
  return model
        
mobilenet_model = create_model(mobilenet_url, num_classes=2)



mobilenet_model.compile(loss='categorical_crossentropy',
                             optimizer=tf.keras.optimizers.Adam(),
                             metrics=['accuracy'])


history = mobilenet_model.fit(training_dataset, epochs=5, steps_per_epoch=len(training_dataset), validation_data=test_data,
                                          validation_steps=len(test_data),
                                          callbacks=[create_tensorboard_callback(dir_name="tensorflow_hub", 
                                                                                 experiment_name="MobileNet_v2")]) 

我在以下行收到錯誤：

mobilenet_model = create_model(mobilenet_url, num_classes=2)

錯誤堆棧跟蹤如下：

ValueError：調用層“feature_extractor_layer”（KerasLayer 類型）時遇到異常。

在用戶代碼中：

    File "/usr/local/lib/python3.7/dist-packages/tensorflow_hub/keras_layer.py", line 237, in call  *
        result = smart_cond.smart_cond(training,

    ValueError: Could not find matching concrete function to call loaded from the SavedModel. Got:
      Positional arguments (4 total):
        * Tensor("inputs:0", shape=(None, 224, 224), dtype=float32)
        * False
        * False
        * 0.99
      Keyword arguments: {}
    
     Expected these arguments to match one of the following 4 option(s):
    
    Option 1:
      Positional arguments (4 total):
        * TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='inputs')
        * True
        * False
        * TensorSpec(shape=(), dtype=tf.float32, name='batch_norm_momentum')
      Keyword arguments: {}
    
    Option 2:
      Positional arguments (4 total):
        * TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='inputs')
        * True
        * True
        * TensorSpec(shape=(), dtype=tf.float32, name='batch_norm_momentum')
      Keyword arguments: {}
    
    Option 3:
      Positional arguments (4 total):
        * TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='inputs')
        * False
        * True
        * TensorSpec(shape=(), dtype=tf.float32, name='batch_norm_momentum')
      Keyword arguments: {}
    
    Option 4:
      Positional arguments (4 total):
        * TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='inputs')
        * False
        * False
        * TensorSpec(shape=(), dtype=tf.float32, name='batch_norm_momentum')
      Keyword arguments: {}


Call arguments received:
  • inputs=tf.Tensor(shape=(None, 224, 224), dtype=float32)
  • training=None

我想知道如何使用自己的圖像形狀進行特征提取？ 如果不可能，我怎樣才能為特征提取器充分輸入這些尺寸的圖像

Answer 1

您需要IMG_SHAPE = (384, 288)重塑為(224,224)作為 mobilenet_v2 的輸入。 重塑的方法之一是將帶有tf.image.resize的Lambda layer添加到您的模型中：

def create_model(model_url, num_classes=2):
    inp = tf.keras.layers.Input((384, 288,3))
    resize_img = tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (224,224)))

    feature_extractor_layer = hub.KerasLayer(model_url, trainable=False, 
                                             name="feature_extractor_layer", 
                                             input_shape=(224,224,3))

    model = tf.keras.Sequential([
                                 inp,
                                 resize_img,
                                 feature_extractor_layer, 
                                 tf.keras.layers.Dense(num_classes, 
                                                       activation="softmax", 
                                                       name="output_layer")
                                 ])
    return model

示例代碼：（您可以在此處閱讀另一個示例） ：

import numpy
from PIL import Image
import tensorflow as tf
import tensorflow_hub as hub
from tensorflow.keras.preprocessing.image import ImageDataGenerator

for loc, rep in zip(['training', 'test'], [20,10]):
    for idx, c in enumerate([f'c/{loc}/1/', f'c/{loc}/2/']*rep):
        array = numpy.random.rand(384,288,3) * 255
        img = Image.fromarray(array.astype('uint8')).convert('RGB')
        img.save('{}img_{}.png'.format(c, idx))

IMG_SHAPE = (384, 288)
BATCH_SIZE = 32

train_dir = 'c/training'
test_dir = 'c/test'


train_datagen = ImageDataGenerator(rescale=1/255.)
test_datagen = ImageDataGenerator(rescale=1/255.)


training_dataset = train_datagen.flow_from_directory(train_dir, target_size=IMG_SHAPE,
                                                     batch_size=BATCH_SIZE, class_mode='categorical')


test_dataset = test_datagen.flow_from_directory(test_dir, target_size=IMG_SHAPE,
                                             batch_size=BATCH_SIZE, class_mode='categorical')
    
mobilenet_url = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4"



def create_model(model_url, num_classes=2):
    inp = tf.keras.layers.Input((384, 288,3))
    resize_img = tf.keras.layers.Lambda(lambda image: tf.image.resize(image, (224,224)))

    feature_extractor_layer = hub.KerasLayer(model_url, trainable=False, 
                                             name="feature_extractor_layer", 
                                             input_shape=(224,224,3))

    model = tf.keras.Sequential([
                                 inp,
                                 resize_img,
                                 feature_extractor_layer, 
                                 tf.keras.layers.Dense(num_classes, 
                                                       activation="softmax", 
                                                       name="output_layer")
                                 ])
    return model
        

mobilenet_model = create_model(mobilenet_url, num_classes=3)
mobilenet_model.compile(loss='categorical_crossentropy',optimizer=tf.keras.optimizers.Adam(),metrics=['accuracy'])
history = mobilenet_model.fit(training_dataset, epochs=5, steps_per_epoch=len(training_dataset),
                              validation_data=test_dataset,validation_steps=len(test_dataset))

輸出：

Found 40 images belonging to 3 classes.
Found 20 images belonging to 3 classes.
Epoch 1/5
2/2 [==============================] - 18s 7s/step - loss: 0.9844 - accuracy: 0.5000 - val_loss: 0.8181 - val_accuracy: 0.5500
Epoch 2/5
2/2 [==============================] - 5s 4s/step - loss: 0.7603 - accuracy: 0.5250 - val_loss: 0.7505 - val_accuracy: 0.4500
Epoch 3/5
2/2 [==============================] - 4s 2s/step - loss: 0.7311 - accuracy: 0.4750 - val_loss: 0.7383 - val_accuracy: 0.4500
Epoch 4/5
2/2 [==============================] - 2s 1s/step - loss: 0.7099 - accuracy: 0.5250 - val_loss: 0.7220 - val_accuracy: 0.4500
Epoch 5/5
2/2 [==============================] - 2s 1s/step - loss: 0.6894 - accuracy: 0.5000 - val_loss: 0.7162 - val_accuracy: 0.5000