如何处理 tf.keras 中输出标签的“无效参数错误”

Question

I am using a data set with classification labels 1 to 8. While using keras sequential model, when I create an output layer with 8 neurons, it gives an Invalid argument error. My understanding is that it recognizes labels as 0 to 7 and does not include 8. Thus, when I create an output layer with 9 neurons, it seems to work. My query is, is it okay to use 9 neurons instead of 8?

Code is as follows:

model = keras.models.Sequential()
model.add(keras.layers.InputLayer(input_shape=X.shape[1:] )) 
model.add(keras.layers.Dense(70, activation="selu",kernel_initializer="lecun_normal"))
model.add(keras.layers.Dense(70, activation="selu",kernel_initializer="lecun_normal"))
model.add(keras.layers.Dense(8, activation="softmax"))

model.compile(loss='sparse_categorical_crossentropy',
              optimizer='sgd',
              metrics=['accuracy'])
history = model.fit(X, y, epochs=100,
                    validation_data=(X_test, y_test))

error message: InvalidArgumentError: Received a label value of 8 which is outside the valid range of [0, 8). Label values: 2 7 7 6 8 1 2 8 6 3 6 8 2 6 1 2 5 8 8 8 1 1 7 8 2 8 6 8 7 5 8 6 [[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at :6) ]] [Op:__inference_train_function_823581]

Answer 1

Using 9 neurons wouldn't be a right choice to go with. Since, you have a total of 8 class labels in your dataset, your logit vector should be 8-dimensional which would then be converted to a categorical distribution by applying the softmax function and thus computing the Cross-entropy loss. The computation of the softmax categorical distribution includes a normalization factor which is the sum of the exponential of the logit vector components. Thus, adding 9 neurons would simply make this computation flawed since you would be adding an extra random value every time.

In this case, the best solution would be to adjust your labels in the pre-processing stage. It is a pretty simple job to do and doesn't require much coding either. Let's say your labels are stored in a numpy array, you could do something like this:

labels = np.array([1, 2, 3, 4, 1, 5, 8]

map_dict = {1:0, 2:1, 3:2, 4:3, 5:4, 6:5, 7:6, 8:7}

for k, v in map_dict.items():
    labels[labels == k] = v

Answer 2

When you have 8 layers at the top of the model and you are using sparse categorial cross entropy loss it expects the label values to be 0 up to 7. When it gets a label of 8 it creates an error. When you change from 8 neurons to 9 it expects to see labels in the range of 0 to 8 so it will not generate an error. Since none of the samples have a label of 0 your network will classify properly.