为什么我的 Tensorflow Keras model output 在训练时会出现奇怪的损失和准确度值？

Question

I have trained a custom text classifier in Tensorflow with python for classifying sentences into questions/sentences containing information using this code:

import tensorflow as tf
from tensorflow import keras


from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

text = ""
with open("/content/train_new.txt") as source:
  for line in source.readlines():
    text = text + line

print("text: " + text)

sentences = []
labels = []

for item in text.split("<n>"):
  parts = item.split("<t>")
  print(parts)
  sentences.append(parts[0])
  labels.append(parts[1])

print(sentences)
print(labels)

print("----")

train_test_split_percentage = 80

training_size = round((len(sentences)/100)*train_test_split_percentage)

print("training size: " + str(training_size) + " of " + str(len(labels)))

training_sentences = sentences[0:training_size]
testing_sentences = sentences[training_size:]

training_labels = labels[0:training_size]
testing_labels = labels[training_size:]

vocab_size = 100
max_length = 10

tokenizer = Tokenizer(num_words = vocab_size, oov_token="<OOV>")
tokenizer.fit_on_texts(sentences)

word_index = tokenizer.word_index

training_sequences = tokenizer.texts_to_sequences(training_sentences)
training_padded = pad_sequences(training_sequences, maxlen=max_length, padding="post", truncating="post")

testing_sequences = tokenizer.texts_to_sequences(testing_sentences)
testing_padded = pad_sequences(testing_sequences, maxlen=max_length, padding="post", truncating="post")

# convert training & testing data into numpy array
# Need this block to get it to work with TensorFlow 2.x
import numpy as np
training_padded = np.array(training_padded)
training_labels = np.asarray(training_labels).astype('float32').reshape((-1,1))
testing_padded = np.array(testing_padded)
testing_labels = np.asarray(testing_labels).astype('float32').reshape((-1,1))

# defining the model
model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, 24, input_length=max_length),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(24, activation='relu'),
    tf.keras.layers.Dense(1, activation='softmax')
])
model.compile(loss='binary_crossentropy',optimizer='adam',metrics=['accuracy'])

# training the model
num_epochs = 1000
history = model.fit(training_padded, training_labels, epochs=num_epochs, validation_data=(testing_padded, testing_labels), verbose=2)

However, while training, it prints weird accuracy and loss values like this:

Epoch 972/1000
9/9 - 0s - loss: -8.2316e+03 - accuracy: 0.7345 - val_loss: -2.7299e+04 - val_accuracy: 0.0000e+00
Epoch 973/1000
9/9 - 0s - loss: -8.2452e+03 - accuracy: 0.7345 - val_loss: -2.7351e+04 - val_accuracy: 0.0000e+00
Epoch 974/1000
9/9 - 0s - loss: -8.2571e+03 - accuracy: 0.7345 - val_loss: -2.7363e+04 - val_accuracy: 0.0000e+00
Epoch 975/1000
9/9 - 0s - loss: -8.2703e+03 - accuracy: 0.7345 - val_loss: -2.7416e+04 - val_accuracy: 0.0000e+00

The train_new.txt file contains data in the form of text<t>class_num<n>

When trying to predict using the model.predict() function, it always outputs [[1.]]

What's the issue with my code?

Answer 1

tf.keras.layers.Dense(1, activation='sigmoid')

You should use sigmoid as activation if you are doing a binary classification. However also,

tf.keras.layers.Dense(2, activation='softmax') 

will be correct in the terms of probability.

Softmax outputs' sum will always be equal to one. That's why you get 1 as output everytime.