Keras training model with images

Question

My first go at training a model on a dataset, this is the data from a pandas dataset

In [16]: exists.head()
Out[16]: 
                 id                                                url  \
1  0a58358a2afd3e4e  http://lh6.ggpht.com/-igpT6wu0mIA/ROV8HnUuABI/...   
2  6b2bb500b6a38aa0  http://lh6.ggpht.com/-vKr5G5MEusk/SR6r6SJi6mI/...   
3  b399f09dee9c3c67  https://lh3.googleusercontent.com/-LOW2cjAqubA...   
4  19ace29d77a5be66  https://lh5.googleusercontent.com/-tnmSXwQcWL8...   
5  2c9c54b62f0a6a37  https://lh5.googleusercontent.com/-mEaSECO7D-4...   
   landmark_id  exists                              filename  
1         6651    True  training_images/0a58358a2afd3e4e.jpg  
2        11284    True  training_images/6b2bb500b6a38aa0.jpg  
3         8429    True  training_images/b399f09dee9c3c67.jpg  
4         6231    True  training_images/19ace29d77a5be66.jpg  
5        10400    True  training_images/2c9c54b62f0a6a37.jpg 

it shows the training image in filename and the classification name in landmark_id

This is the way I've written the model to train it

from keras.preprocessing.image import ImageDataGenerator, array_to_img, img_to_array, load_img
from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
from keras.callbacks import EarlyStopping
from keras.callbacks import ModelCheckpoint

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(3, 150, 150)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(5))
model.add(Dense(activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-3, patience=5, verbose=0, mode='auto')
checkpointer = ModelCheckpoint(filepath="best_weights.hdf5", verbose=0, save_best_only=True) # save best model

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              callbacks=[monitor,checkpointer],
              verbose=0,
              epochs=1000,
              metrics=['accuracy'])

batch_size = 16

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(
        rescale=1./255,
        shear_range=0.2,
        zoom_range=0.2,
        horizontal_flip=True)

# this is the augmentation configuration we will use for testing:
# only rescaling
test_datagen = ImageDataGenerator(rescale=1./255)

# this is a generator that will read pictures found in
# subfolers of 'data/train', and indefinitely generate
# batches of augmented image data
train_generator = train_datagen.flow_from_directory(
        'training_images',  # this is the target directory
        target_size=(150, 150),  # all images will be resized to 150x150
        batch_size=batch_size,
        class_mode='binary')  # since we use binary_crossentropy loss, we need binary labels

# this is a similar generator, for validation data
validation_generator = test_datagen.flow_from_directory(
        'test_images',
        target_size=(150, 150),
        batch_size=batch_size,
        class_mode='binary')

model.fit_generator(
        train_generator,
        steps_per_epoch=2000 // batch_size,
        epochs=50,
        validation_data=validation_generator,
        validation_steps=800 // batch_size)
model.load_weights('best_weights.hdf5') # load weights from best model
model.save('last_model.h5')

I don't know how I'm supposed to put the labels to the image while training. So when it trains and loops through the images in the training_images folder.

Answer 1

Samuel,

Your FitGenerator is getting the training input labels from the flow_from_directory method. This method uses the folder structure to determine the training categories. Since your class is binary, and you have a single sigmoid output, I'm assuming that you are doing a Hot Dog - Not Hot Dog type of classification where you just want a single probability value.

The other hint for me that you care about the single probability that something is a category or not is that your loss function is binary_crossentropy.

Check your training data folder. Look at how the data is organized. This should be set up such that the data is organized correctly.

You seem to be hinting that you want to create multiple labels (eg, car, boat, train). If this is the case, you will want to create those folders under train and validate and put the images in the respective folder. You will need to change several things about your model if you do this though. Your loss, output layer size, and output layer activations will change accordingly.