dnn model 中的过拟合

Question

I have a dataset on which I train a DNN model. my dataset contain 398 samples and 330 features, i redueced features to 39 with ExtraTreeclassifier(). this my model:

X_train, X_test, y_train, y_test = train_test_split(xfinal, val_y, test_size = 0.2, random_state = 0)
model=Sequential()
model.add(Dense(units=20, kernel_initializer='uniform', activation='relu',input_dim=nb_features))
model.add(Dense(units=20, kernel_initializer='uniform', activation='relu'))
model.add(Dense(units=10, kernel_initializer='uniform', activation='relu'))
model.add(Dense(units=5, kernel_initializer='uniform', activation='relu'))
model.add(Dense(units=1,kernel_initializer='uniform',activation='sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
history = model.fit(X_train,y_train,validation_data=(X_test,y_test),batch_size=32,epochs=250)

I tried Dropout but my model steel overfitting:

Any solution for my model?

Answer 1

You can add Dropout layer between Dense layers like below.

model.add(Dropout(0.2))

Also you can remove one or more hidden layers from your architecture.

One more thing is, you can use Earlystopping method to stop at correct epoch number.

Your final model architecture can be like below:

callbacks = [EarlyStopping(monitor='val_loss', patience=5)]
model=Sequential()
model.add(Dense(units=20, kernel_initializer='uniform', activation='relu',input_dim=nb_features))
model.add(Dropout(0.2))
model.add(Dense(units=5, kernel_initializer='uniform', activation='relu'))
model.add(Dense(units=1,kernel_initializer='uniform',activation='sigmoid'))
model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])
history = model.fit(X_train,y_train,validation_data=(X_test,y_test),batch_size=32,epochs=250, callbacks=callbacks)