Fine-Tuning InceptionV3

Question

I want to fine-tuning Inception V3 for recognize the UC Merced Land Use Dataset. It contains 21 classes,with 100 images for each class. Manually I have split the Datataset in 5 fold, for each fold and for each class I have 60 images for training 20 for validation and 20 for Testing. Example: In the first fold for each class the images between 0 and 59 are for Training, the images between 60 and 79 are for Validation ecc... In the second fold the images between 0 and 19 are for Testing,the images between 80 and 99 are for Validation ecc... I applied the cross validation, so at the end I will test the Net with all images in the Dataset.

With this fine-tuning i have reached 93%, 97% is the goal.

# import
from keras.preprocessing import image
import os
from matplotlib import pyplot as plt
import numpy as np
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential, Model
from keras.layers import Conv2D, MaxPooling2D, Activation, Dropout, Flatten, Dense, GlobalAveragePooling2D
from keras import backend as K
from keras import applications
from keras import utils
from keras import optimizers
from sklearn.model_selection import KFold
import random
from keras.preprocessing import image
import os
from matplotlib import pyplot as plt


directory_train=["./UCMerced_LandUse2/Images/Fold1/Training","./UCMerced_LandUse2/Images/Fold2/Training","./UCMerced_LandUse2/Images/Fold3/Training","./UCMerced_LandUse2/Images/Fold4/Training","./UCMerced_LandUse2/Images/Fold5/Training"]
directory_validation=["./UCMerced_LandUse2/Images/Fold1/Validation","./UCMerced_LandUse2/Images/Fold2/Validation","./UCMerced_LandUse2/Images/Fold3/Validation","./UCMerced_LandUse2/Images/Fold4/Validation","./UCMerced_LandUse2/Images/Fold5/Validation"]
directory_test=["./UCMerced_LandUse2/Images/Fold1/Test","./UCMerced_LandUse2/Images/Fold2/Test","./UCMerced_LandUse2/Images/Fold3/Test","./UCMerced_LandUse2/Images/Fold4/Test","./UCMerced_LandUse2/Images/Fold5/Test"]

img_width, img_height = 256, 256

num_samples=2100
batch_size = 10

Datagen = ImageDataGenerator(rescale=1./255,
    shear_range=0.2,        
    zoom_range=0.2,           
    horizontal_flip=True)

Gen = ImageDataGenerator(rescale=1./255)

train_generator=[]
valid_generator=[]
test_generator=[]

print("Creazione Train Generator")

for i in range(5):
  train_generator.append(Datagen.flow_from_directory(
  directory_train[i],
  target_size=(img_width, img_height),  
  batch_size=batch_size, 
  class_mode='categorical'
   ))

print(train_generator[0].n//batch_size)
print("Creazione Validation Generator")

for i in range(5):
  valid_generator.append(Gen.flow_from_directory(
  directory_validation[i],
  target_size=(img_width, img_height),  
  batch_size=batch_size, 
  class_mode='categorical'
   ))

print(valid_generator[i].n//batch_size)

print("Creazione Test Generator")

for i in range(5):
  test_generator.append(Gen.flow_from_directory(
  directory_test[i],
  target_size=(img_width, img_height),  
  batch_size=batch_size, 
  class_mode='categorical'
   ))

# Inception V3 with pre-trained weights
base_model = applications.InceptionV3(weights='imagenet', include_top=False,input_shape=(256,256,3),classes=21)
base_model.trainable=True;


num_epochs = 100
history=[]
risultati=[]


for i in range(5):

  model=Sequential()  
  model.add(base_model)
  model.add(GlobalAveragePooling2D())
  model.add(Dense(1024,activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(1024,activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(100,activation='relu'))
  model.add(Dropout(0.5))
  model.add(Dense(21,activation='softmax'))
  model.summary()

  model.compile(loss='categorical_crossentropy',
              optimizer=optimizers.SGD(1e-4,momentum=0.9),
              metrics=['accuracy'])

  print(i)
  model.fit_generator(train_generator[i],steps_per_epoch=train_generator[i].n//batch_size,epochs=num_epochs,validation_data=valid_generator[i],validation_steps=valid_generator[i].n//batch_size,shuffle=True)
  model.save('sasaprova.h5')
  print(i)
  scores=model.evaluate_generator(test_generator[i])
  print(scores[1])
  risultati.append(scores[1]*100)


print(np.mean(risultati))

any suggestions?

Answer 1

There are few that may improve the classification accuracy:

1. Use EfficientNet with noisy_student weights. There are less number of parameters to train. It gives better accuracy due to the scalable architecture it has.

2. You can use test time augmentation. In your test data generator, do a simple horizontal flip, vertical flip (if data looks realistic) and affine transformations. It will generate multiple views of the data and helps the model to average out more probable class.

3. Your data augmentation could be more exhaustive. Checkout imgaug library. Plus, there are random_eraser, cut out and mix up strategies that have been proved to be useful.

4. Try label smoothing. It can also help your classifier to give more probability to the correct class.

5. Try learning rate warmup.