如何将数值数据输入 Tensorflow ResNet50 model 进行回归？

Question

我目前有一个 TensorFlow model（ResNet50），它采用单个图像输入并通过回归输出连续值（范围从 0.8 - 2.0）。 该数据集有 3000 名不同的患者，每个患者都有一张图像和几个数字数据点（特别是年龄、性别、体重）。 我可以通过对图像进行训练来获得不错的准确性，但我想知道如何将数字数据点添加为单独的输入。 数值数据位于 csv 文件中，其中每一行是单独的患者，不同的列包含不同的值。 这是我对 1 张图片的看法：

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.applications.resnet import ResNet50
from tensorflow.keras.applications.resnet_rs import ResNetRS50
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import GlobalAveragePooling2D, Dense, Dropout, Flatten, GlobalMaxPooling2D
from tensorflow.keras.optimizers import Adam
import warnings

labels = pd.read_csv('[path goes here]')
labels.head()

def load_train(path):
    
    """
    It loads the train part of dataset from path
    """
    #1 / 255
    labels = pd.read_csv('[Target Dataframe path]')
    train_datagen = ImageDataGenerator(validation_split=0.2, rescale=None)
    train_gen_flow = train_datagen.flow_from_dataframe(
        dataframe=labels,
        directory='[Images folder path]',
        x_col='ID',
        y_col='Value',
        target_size=(224, 224),
        batch_size=32,
        class_mode='raw',
        subset = 'training',
        seed=1234)

    return train_gen_flow


def load_test(path):
    
    """
    It loads the validation/test part of dataset from path
    """
    labels = pd.read_csv('[Target Dataframe path]')
    validation_datagen = ImageDataGenerator(validation_split=0.2, rescale=None)
    test_gen_flow = validation_datagen.flow_from_dataframe(
    dataframe = labels,
    directory='[Images folder path]',
    x_col="ID",
    y_col="Value", 
    class_mode="raw", 
    target_size=(224,224), 
    batch_size=32,
    subset = "validation",
    seed=1234,
    )


    return test_gen_flow


def create_model(input_shape):
    
    """
    It defines the model
    """

    backbone = ResNetRS50(input_shape=input_shape, weights='imagenet', include_top=False)
    model = Sequential()
    model.add(backbone)
    
    model.add(Dropout(0.3))
    model.add(GlobalMaxPooling2D())
    
    model.add(Dense(1, activation='linear'))
    optimizer = Adam(learning_rate=0.0003)
    model.compile(optimizer=optimizer, loss='mae', metrics=['mae'])
    print(model.summary())

    return model


def train_model(model, train_data, test_data, batch_size=32, epochs=100,
                steps_per_epoch=None, validation_steps=None):

    """
    Trains the model given the parameters
    """
    history = model.fit(train_data, validation_data=test_data, batch_size=batch_size, 
              epochs=epochs, steps_per_epoch=steps_per_epoch, 
              validation_steps=validation_steps, verbose=2)

    # Get training and test loss histories
    training_loss = history.history['loss']
    test_loss = history.history['val_loss']

# Create count of the number of epochs
    epoch_count = range(1, len(training_loss) + 1)

# Visualize loss history
    plt.plot(epoch_count, training_loss, 'r--')
    plt.plot(epoch_count, test_loss, 'b-')
    plt.legend(['Training Loss', 'Test Loss'])
    plt.xlabel('Epoch')
    plt.ylabel('Loss')
    plt.show();

    return model

path = '[Full set path (contains image folder, target csv, and numerical feature csv]'

train_data = load_train(path)
test_data = load_test(path)

#build a model
model = create_model(input_shape = (224, 224, 3))

model = train_model(model, train_data, test_data)

Answer 1

您可以使用功能性 API，它为您提供更多灵活性（多个输入或输出，跳过层之间的连接等）。 在您的情况下，您可能希望有两个单独的输入。 对数字特征进行编码，将它们传递给一个密集层，然后在最后一个密集层之前将它们添加到您的主网络中。

https://www.tensorflow.org/guide/keras/functional

https://machinelearningmastery.com/keras-functional-api-deep-learning/