简体   繁体   中英

Neural network not converging on clear linearly seperable problem

I am developing a neural network in Keras and I want to test to make sure it works. The feature set is obviously linearly seperable ('A' and 'N' in the below figure) but for some reason when I run my neural network using heart rate variability (HRV in the figure) as the sole feature, it doesn't think the positive training examples ('A' training examples) are unique:

明显线性可分离

My neural network architecture is a simple one:

model = Sequential()

model.add(Dense(10, input_shape=(None, X_train.shape[1]),
           activation='sigmoid'))

model.add(Dense(10, activation='sigmoid'))

model.add(Dense(1, activation='sigmoid'))

opt = tf.keras.optimizers.Adam(lr=learning_rate, decay=decay_rate)

model.compile(loss=loss_fn, optimizer=opt, metrics=['acc'])

model.fit(X_train, y_train, epochs=n_epochs, validation_data=(X_test, y_test))

When I test the accuracy using a confusion matrix, the NN overfits to the negative training examples:

          precision    recall  f1-score   support

     0.0       0.72      1.00      0.84     20774
     1.0       0.00      0.00      0.00      8126

accuracy                           0.72     28900
macro avg       0.36      0.50      0.42     28900
weighted avg       0.52      0.72      0.60     28900

Any suggestions?

Edit: Additional hyperparamters

Training vector shape:(67432, 1, 1)
Example of first element:[[72.710655]

loss_fn = 'binary_crossentropy'
learning_rate = 1e-2
decay_rate = 1e-8
n_epochs = 10 (have varied this but still converges to negative training example)

Edit: Additional information

I wanted to include how I formatting my arrays in case the issue is with that:

X_train = np.asarray(X_train).reshape(
    X_train.shape[0], 1, X_train.shape[1])

X_test = np.asarray(X_test).reshape(
    X_test.shape[0], 1, X_test.shape[1])

Since you have imbalanced data you may want to use class_weight in model.fit

model.fit(X_train, y_train, epochs=n_epochs, validation_data=(X_test, y_test), class_weight=class_weight)

Here class_weight is a dictionary. One option you can try is to set the class weight inversely proportional to the class size, ie the number of data points of each class.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM