Keras NN中的恒定精度/损失

Question

I am learning deep learning and was implementing the titanic challenge on kaggle.

In Preprocessing,I have dropped the Pid,name,ticket and cabin columns.I have replaced the empty values in Fare and Age columns by adding the respective means. I have one hot encoded sex, Pclass(passenger class) and Embarked Port. Added an alone column to determine if the passenger was traveling alone or not(From sibling and parents count). Finally I used scikit's MinMaxScalar.

Here is the keras model used.

model = Sequential()
model.add(Dense(32, input_dim=13, activation="relu"))
model.add(Dense(64, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(256, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(512, activation="relu"))
model.add(Dense(1024, activation="relu"))
model.add(Dense(512, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(1024, activation="relu"))
model.add(Dense(2048, activation="relu"))
model.add(Dense(1024, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(512, activation="relu"))
model.add(Dense(1024, activation="relu"))
model.add(Dense(512, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(256, activation="relu"))
model.add(Dense(128, activation="relu"))
model.add(Dense(64, activation="relu"))
model.add(Dense(32, activation="relu"))
model.add(Dropout(0.3))
model.add(Dense(1, activation="sigmoid"))
model.compile(optimizer="Adam", loss='binary_crossentropy', metrics=["binary_accuracy"])

The val_binary_accuracy is a constant 0.6425 throughout the process

Answer 1

This is common with "relu".

"Relu" has a "constant zero" region. If any of the layer happens to fall entirely in this region, your model is dead.

You can try one of these:

• Create the model again (if you have a random seed, change the seed)
• Reduce the learning rate (but you don't seem to be using a really big one)
• Use different activations ('tanh', 'sigmoid', etc.)
• Use BatchNormalization after the layer and before relu .