尝试为 CIFAR-10 创建一个完全连接的神经网络

Question

I am a relative beginner when it comes to machine learning.

I have been playing with Keras with TensorFlow as a backend and for some reason I am not getting good accuracy when I am using the CIFAR-10 dataset.

This is my code.

model = Sequential()

batch_size = 250

model.add(Dense(100, input_shape = (3072, ), activation='relu',
bias_initializer = 'RandomNormal',kernel_regularizer=regularizers.l2(.01)))


model.add(Dense(50))

model.add(Dense(10))

model.compile(optimizer=keras.optimizers.SGD(lr=0.004), 
loss='hinge', metrics=['categorical_accuracy'])

model.fit(x=X_train, y=utils.to_categorical(Y_Train, num_classes = 10), 
batch_size = batch_size, epochs = 100, validation_split = .4)

X_Train is a (50000, 3072) numpy array and Y_Train is a (50000, 1) numpy array.

The result I got was

loss: 1.1865

categorical_accuracy: 0.1696

val_loss: 1.1859

val_categorical_accuracy: 0.1668

in 100 epochs.

My setup is Ubuntu 18.04, Python 3.6, Numpy 1.16, Keras 2.2.4

Is there something wrong in my code or is it the fact that a fully connected neural network is just a bad setup for image classification and one should use a convolution neural network?

Answer 1

There are a number of issues with your model:

• Layers 2 and 3 have no activation, and are thus linear (useless for classification, in this case)

• Specifically, you need a softmax activation on your last layer. The loss won't know what to do with linear output.

• You use hinge loss, when you should be using something like categorical_crossentropy .

What Jibin said about your fully connected model not being complex enough is not true, you don't need much complexity to get decent accuracy on CIFAR10.