训练多输出Keras模型

Question

I have 10,000 images, each of which are labeled with 20 tags. For each image, the tag is either true or false. I'm trying to train a multi-output model to perform all these 20 binary classifications with one network.

The network is a Residual Network. After the flatten layer, the network branches out into 20 branches. Each branch has 2 fully connected layers, each of which are followed by a drop out layer. And finally a dense layer with one node and sigmoid activation in the end.

The labels for each image and the image name are stored in a text file, for both train and validation set. Like this: 1.jpg 1 -1 1 -1 -1 1 -1.........

I wrote my own generator, but I can't get them to work. I keep getting this error:

Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 20 array(s), but instead got the following list of 1 arrays.

Function explanations: get_input function reads an image and resizes it. get_output prepares the labels for each image. The labels are stored in a list and returned in the end. preprocess_input performs preprocessing and converting images into arrays. train_generator and validation_generator generate batches with size 32 to be fed to the model.

Here's my code:

def get_input(img_name):
    path = os.path.join("images", img_name)
    img = image.load_img(path, target_size=(224, 224))

    return img


def get_output(img_name, file_path):
    data = pd.read_csv(file_path, delim_whitespace=True, header=None)

    img_id = img_name.split(".")[0]
    img_id = img_id.lstrip("0")
    img_id = int(img_id)

    labels = data.loc[img_id - 1].values
    labels = labels[1:]

    labels = list(labels)
    label_arrays = []
    for i in range(20):
        val = np.zeros((1))
        val[0] = labels[i]
        label_arrays.append(val)

    return label_arrays


def preprocess_input(img_name):
    img = get_input(img_name)
    x = image.img_to_array(img)
    x = np.expand_dims(x, axis=0)    
    return x

def train_generator(batch_size):
    file_path = "train.txt"
    data = pd.read_csv(file_path, delim_whitespace=True, header=None)

    while True:
        for i in range(math.floor(8000/batch_size)):
            x_batch = np.zeros(shape=(32, 224, 224, 3))
            y_batch = np.zeros(shape=(32, 20))
            for j in range(batch_size):
                img_name = data.loc[i * batch_size + j].values
                img_name = img_name[0]
                x = preprocess_input(img_name)
                y = get_output(img_name, file_path)
                x_batch[j, :, :, :] = x
                y_batch[j] = y
            yield(x_batch, y_batch)

def val_generator(batch_size):
    file_path = "val.txt"
    data = pd.read_csv(file_path, delim_whitespace=True, header=None)

    while True:
        for i in range(math.floor(2000/batch_size)):
            x_batch = np.zeros(shape=(32, 224, 224, 3))
            y_batch = np.zeros(shape=(32, 20))
            for j in range(batch_size):
                img_name = data.loc[i * batch_size + j].values
                img_name = img_name[0]
                x = preprocess_input(img_name)
                y = get_output(img_name, file_path)
                x_batch[j, :, :, :] = x
                y_batch[j] = y
            yield(x_batch, y_batch)

Edit: One quick question. What's the difference between this loop and the one in your answer:

ys = []
for i in range(batch_size):
    ys.append(y_batch[i, :])

yield(x_batch, ys)

Answer 1

If your model has 20 outputs then you must provide a list of 20 arrays as target. One way of doing this is to modify the generator (for both training and validation):

ys = []
for i in range(20):
    ys.append(y_batch[:,i])

yield(x_batch, ys)

As a side note, you mentioned that you have 20 tags per sample then why have you specified 40 in the input shape?

y_batch = np.zeros(shape=(32, 40))

Further, I don't know about the specific problem you are working on but alternatively you could only have one output of size 20 instead of 20 outputs with size one.

Answer 2

You can test the generator output dimensions initializing the generator and call the function next() to check the dimensions. For example with the train_generator:

train_gen = train_generator(batch_size)
x_batch, y_batch = next(train_gen)

Then check x_batch and y_batch dimensions and datatype

I would make the generator in this way:

def train_generator(batch_size):
    file_path = "train.txt"
    data = pd.read_csv(file_path, delim_whitespace=True, header=None)
    # Initialize empty list
    x_batch = []
    y_batch = []

    while True:
        for i in range(math.floor(8000/batch_size)):
            for j in range(batch_size):
                img_name = data.loc[i * batch_size + j].values
                img_name = img_name[0]
                x = preprocess_input(img_name)
                y = get_output(img_name, file_path)
                x_batch.append(x)
                y_batch.append(y)

            yield(np.array(x_batch), np.array(y_batch))