Keras错误中的flow_from_dataframe方法：目标数据丢失，

Question

I want to do an image segmentation having the following structure folder

--img 
    -------0001.slice.png 
--mask
    -------0001.slice.png

I try to make a dataframe, and also ImageDataGenerator .

X_train_data consist of a list of all the absolute path to all the image files, while the Y_train_label is also the same, consisting of a list of all absolute path of the masks. Then I make a dictionary to bound them altogether

data = OrderedDict()                    

data = {"image":X_train_data, "label":Y_train_label}
df = pd.DataFrame(data=data)  

then I put them into the ImageDatagenerator with flow_from_dataframe

train_data_gen=train_datagen.flow_from_dataframe(dataframe=training_data,
                                                 directory=None,
                                                 x_col="image", 
                                                 y_col="label",
                                                 validate_filenames=(True),
                                                 color_mode='rgb',
                                                 target_size=(IMAGE_HEIGHT, IMAGE_WIDTH),
                                                 batch_size=32, 
                                                 class_mode=None,
                                                 shuffle=(False))

then I fit model as below:

history = model.fit(train_data_gen, 
                    steps_per_epoch=train_data_gen.n//train_data_gen.batch_size,   
                    validation_data= validation_data_gen,
                    validation_steps=validation_data_gen.n//validation_data_gen.batch_size,epochs=5)

but then I get the error

ValueError: Target data is missing. Your model was compiled with loss=binary_crossentropy, and therefore expects target data to be provided in fit().

Why the y_col which contain all the absolute path can not be read by that flow_from_dataframe ?

I have tried to change the Y_train_data value by storing all the NumPy array of the masks but still get the same error. I also have used the keras-preprocessing from GitHub.

Answer 1

history = model.fit(generator=train_data_gen, 
                    steps_per_epoch=train_data_gen.n//train_data_gen.batch_size,   
                    validation_data=validation_data_gen,
                    validation_steps=validation_data_gen.n//validation_data_gen.batch_size,
                    epochs=5)

Answer 2

This probably occurs because you are using labels named as Y_train_data then raises ValueError because is it wrong, i suggest to structure your data files to perform a "clean" training enviroment.

/img
-------0001.slice.png
/mask
-------0001.slice.png

import os
from random import randint

import pandas 

input_dir = "img/"
target_dir = "mask/" 
img_size = (IMAGE_HEIGHT, IMAGE_WIDTH)
num_classes = 2
batch_size = 3 

# replace image format ".bmp" or anoter like jpg, png, etc ...

X_train_data = [image for image in os.listdir(input_dir) if image.endswith(".bmp")]
 
Y_train_label = [image for image in os.listdir(target_dir) if image.endswith(".bmp")] 

labels = [randint(0, 1) for i in range(len(X_train_data))]

df = pd.DataFrame(data=[X_train_data, Y_train_label, labels], columns=["X_train", "y_train", "label"])

At this point it is similar to your example, at least you have to implement in your script

generator = image.ImageDataGenerator()

train_data_gen = generator.flow_from_dataframe(dataframe=df,
                                                directory=None,
                                                x_col="X_train", 
                                                y_col="y_train",
                                                validate_filenames=(True),
                                                color_mode='rgb',
                                                target_size=(IMAGE_HEIGHT, IMAGE_WIDTH),
                                                batch_size=batch_size, 
                                                class_mode=None,
                                                shuffle=(False))

history = model.fit(train_data_gen, 
                steps_per_epoch=train_data_gen.n//train_data_gen.batch_size,   
                validation_data= validation_data_gen,
                validation_steps=validation_data_gen.n//validation_data_gen.batch_size,epochs=5)