神经网络的最后一层应该有多少个神经元？

Question

I use the following code to classify movie reviews into three classes (negative as -1, neutral as 0, and 1 as positive). But is it true that there is only one output neuron in the last layer for the three-class classification problem?

import tensorflow as tf
import numpy as np
import pandas as pd
import numpy as K

csvfilename_train = 'train(cleaned).csv'
csvfilename_test = 'test(cleaned).csv'

# Read .csv files as pandas dataframes
df_train = pd.read_csv(csvfilename_train)
df_test = pd.read_csv(csvfilename_test)

train_sentences  = df_train['Comment'].values
test_sentences  = df_test['Comment'].values

# Extract labels from dataframes
train_labels = df_train['Sentiment'].values
test_labels = df_test['Sentiment'].values

vocab_size = 10000
embedding_dim = 16
max_length = 30
trunc_type = 'post'
oov_tok = '<OOV>'

from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

tokenizer = Tokenizer(num_words = vocab_size, oov_token = oov_tok)
tokenizer.fit_on_texts(train_sentences)
word_index = tokenizer.word_index
sequences = tokenizer.texts_to_sequences(train_sentences)
padded = pad_sequences(sequences, maxlen = max_length, truncating = trunc_type)

test_sequences = tokenizer.texts_to_sequences(test_sentences)
test_padded = pad_sequences(test_sequences, maxlen = max_length)

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length = max_length),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(6, activation = 'relu'),
    tf.keras.layers.Dense(1, activation = 'sigmoid'),
])
model.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])

num_epochs = 10
model.fit(padded, train_labels, epochs = num_epochs, validation_data = (test_padded, test_labels))

When I changes tf.keras.layers.Dense(1, activation = 'sigmoid') to tf.keras.layers.Dense(2, activation = 'sigmoid') it gives me the following error :

---> 10 model.fit(padded, train_labels, epochs = num_epochs, validation_data = (test_padded,test_labels))
     ValueError: logits and labels must have the same shape ((None, 2) vs (None, 1))

Answer 1

You should have 3 neurons if you are classifying between 3 categories.

Also, you should use the 'softmax' activation for your final layer, assuming that all observations are in one class only.

Next, you should use 'sparse_categorical_crossentropy' since your input is not one-hot encoded. Targets like [0,0,1], [0,1,0], [1,0,0] are optional, you can also have [1, 2, 0, 1, 2, 1, 0] .

Finally, your targets should be [0, 1, 2] and not [-1, 0, 1] so I suggest you add 1 to your labels.

test_labels = df_test['Sentiment'].values + 1

This is what happens if labels are [-1, 0, 1] instead of [0, 1, 2] :

import tensorflow as tf

sparse_entropy = tf.losses.SparseCategoricalCrossentropy()

a = tf.convert_to_tensor([[-1., 0., 1.]]) #+ 1
b = tf.convert_to_tensor([[.4, .2, .4], [.1, .7, .2], [.8, .1, .1]])

sparse_entropy(a, b)

nan

If you uncomment the +1 , which transforms the labels into [0, 1, 2] , it works:

<tf.Tensor: shape=(), dtype=float32, numpy=1.1918503>

Answer 2

Short answer:

One hot encode your train labels and use categorical crossentropy as loss function.

Cause:

1. Your logits have shape (n,2) but labels have (n,1).
2. Your logits and labels should be of shape (n,3) if youre using crossentropy(unless it is sparse).

Solution:

1. One hot encode the train labels and you'll get train labels shape (n,3)
2. Use categorical crossentropy with final dense neuron having 3 outputs, then you'll get logits shape(n,3)

Your model will start learning after this.

Answer 3

You got 3 classes -> num_classes=3 Your last layer should look like this:

tf.keras.layers.Dense(num_classes, activation = 'sigmoid'),

You will receive a np.array with 3 probabilities as output. Moreover, you should change your class to categorical_crossentropy because you are not solving a binary problem.