级联神经网络架构和输入使用 TensorFlow

Question

我目前正在复制本文中发现的机器学习架构： https://downloads.hindawi.com/journals/acisc/2018/1439312.pdf

特别是在论文的第 4 页：

关于如何在 TensorFlow 上实现此功能的任何想法？ 我当前的 model 看起来像这样：

BATCH_SIZE = 32

INPUT_SIZE1 = (15, 30, 3)
INPUT_SIZE2 = (30, 60, 3)
INPUT_SIZE3 = (40 ,80, 3)

LEARNING_RATE = 0.001
EPOCHS = 10

CNN_CR1_INPUT = keras.Input(shape = INPUT_SIZE1)
CNN_CR1 = Conv2D(64, (5, 5), strides=2, padding='same', activation='relu')(CNN_CR1_INPUT)
CNN_CR1 = MaxPooling2D(3,3)(CNN_CR1)
CNN_CR1 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR1)
CNN_CR1 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR1)
CNN_CR1 = Flatten()(CNN_CR1)
CNN_CR1_OUTPUT = Dense(1)(CNN_CR1)

CNN_CR2_INPUT = keras.Input(shape = INPUT_SIZE2)
CNN_CR2 = Conv2D(64, (5,5), strides=2, padding='same', activation='relu')(CNN_CR2_INPUT)
CNN_CR2 = MaxPooling2D(3, 3)(CNN_CR2)
CNN_CR2 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR2)
CNN_CR2 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR2)
CNN_CR2 = Flatten()(CNN_CR2)
CNN_CR2_OUTPUT = Dense(1)(CNN_CR2)

CNN_CR3_INPUT = keras.Input(shape = INPUT_SIZE3)
CNN_CR3 = Conv2D(64, (5,5), strides=2, padding='same', activation='relu')(CNN_CR3_INPUT)
CNN_CR3 = MaxPooling2D(3, 3)(CNN_CR3)
CNN_CR3 = Conv2D(64, (3,3), strides=1, padding='same', activation='relu')(CNN_CR3)
CNN_CR3 = Conv2D(64, (3,3), strides=2, padding='same', activation='relu')(CNN_CR3)
CNN_CR3 = Flatten()(CNN_CR3)
CNN_CR3_OUTPUT = Dense(1)(CNN_CR3)

# SUGGESTION: This kinda weird. If this works, we only need 1 valitadation datagen? Not sure how it all connect together.
CNN_MAX = Maximum()([CNN_CR1_OUTPUT, CNN_CR2_OUTPUT, CNN_CR3_OUTPUT])

CNN_MODEL = keras.Model(inputs=[CNN_CR1_INPUT, CNN_CR2_INPUT, CNN_CR3_INPUT], outputs=[CNN_MAX])

我不确定我制作的 model 是否正确，我需要一些帮助。 此外，您如何为像这样的级联神经网络创建输入管道？ 我已经尝试过了：

TRAIN_DATAGEN1 = ImageDataGenerator(
    # SUGGESTION: Not sure if this is needed??
    rescale = 1/255.0
)

TRAIN_GENERATOR1 = TRAIN_DATAGEN1.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '0'),
    target_size = (15, 30),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN1 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR1 = TEST_DATAGEN1.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '0'),
    target_size = (15, 30),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

# CNN 2
TRAIN_DATAGEN2 = ImageDataGenerator(
    rescale = 1/255.0
)

TRAIN_GENERATOR2 = TRAIN_DATAGEN2.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '1'),
    target_size = (30, 60),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN2 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR2 = TEST_DATAGEN2.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '1'),
    target_size = (30, 60),
    class_mode ='binary',
    batch_size = BATCH_SIZE
)

# CNN 3
TRAIN_DATAGEN3 = ImageDataGenerator(
    rescale = 1/255.0
)

TRAIN_GENERATOR3 = TRAIN_DATAGEN3.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['train']['images'], '2'),
    target_size = (40 ,80),
    class_mode = 'binary',
    batch_size = BATCH_SIZE
)

TEST_DATAGEN3 = ImageDataGenerator(
    rescale = 1/255.0,
)

TEST_GENERATOR3 = TEST_DATAGEN3.flow_from_directory(
    os.path.join(WORKING_DATASETS['GI4E']['test']['images'], '2'),
    target_size = (40 ,80),
    class_mode = 'binary',
    batch_size = BATCH_SIZE
)

但是当我尝试安装它时它会吐出一个错误。

ValueError：找不到可以处理输入的数据适配器：（<class 'list'> 包含类型 {"<class 'keras.preprocessing.image.DirectoryIterator'>"}），<class 'NoneType'>

深入研究文档后，我知道 TensorFlow 无法在列表中同时处理一堆 ImageDataGenerator。 但我不确定如何在没有 ImageDataGenerator 的情况下将图像提供给 model。

所以，总结一下：

• 如何在 TensorFlow 中重新创建所说的 model？ 我创建的 model 是否已经正确？
• 如何为 model 创建输入管道？ 除了 ImageDataGenerator 之外还有什么选择吗？

Answer 1

You could try building the model with tf.data - load the datas with keras.utils, for data augmentation you can add layers on to model architecture (tf.keras.layers.Rescaling,tf.keras.layers.RandomRotation etc). 下面的文档中提供了其他增强方法，它们在利用 ImageDataGenerator 的 GPU 方面更有效

https://www.tensorflow.org/tutorials/images/data_augmentation#data_augmentation_2 https://www.tensorflow.org/api_docs/python/tf/keras/utils https://www.tensorflow.org/guide/keras/预处理层