当我们使用实值训练目标时，为什么使用二进制交叉熵损失函数进行的神经网络训练会停滞不前？

Question

Assume that we have a binary classification problem, in which the training targets are not in {0,1} but in [0,1]. We use the following code to train a simple classifier in Keras:

model = Sequential()
model.add(Dense(100, input_shape=(X.shape[1],), activation='relu'))
model.add(Dense(1, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='rmsprop')
model.fit(X,y)

If we pass the real training targets (in [0,1]), the training hardly proceeds, getting stuck around its initial loss value; but if we quantize the targets in {0,1} it performs better, rapidly decreases the training loss.

Is this a normal phenomena? What is its reason?

Edit: Here is the reproducible experiment. And this is the obtained plot:

Answer 1

You state that you want to solve a binary classification task, for which the target should be binary -valued, ie {0,1}.

However, if your target instead is some float value in [0,1], you are actually trying to perform regression .

This, amongst others, changes the requirements for your loss function. See Tensorflow Cross Entropy for Regression? , where the usage of cross entropy loss for regression is discussed in more detail.