Tensorflow 估计器 ValueError：logits 和标签必须具有相同的形状 ((?, 1) vs (?,))

Question

I'm classifying movie reviews as positive or negative using binary crossentropy. So, when I'm trying to wrap my keras model with tensorflow estimator, I get the error:

Tensorflow estimator ValueError: logits and labels must have the same shape ((?, 1) vs (?,))

I'm using sigmoid activation as my last layer, guess I'm missing something trivial here. Any help?

from tensorflow import keras
import tensorflow as tf
print("Tensorflow {} loaded".format(tf.__version__))
import numpy as np

keras.__version__
from keras.datasets import imdb

(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
def vectorize_sequences(sequences, dimension=10000):
    # Create an all-zero matrix of shape (len(sequences), dimension)
    results = np.zeros((len(sequences), dimension))
    for i, sequence in enumerate(sequences):
        results[i, sequence] = 1.  # set specific indices of results[i] to 1s
    return results.astype('float32')

# Our vectorized training data
x_train = vectorize_sequences(train_data)

# Our vectorized test data
x_test = vectorize_sequences(test_data)

# Our vectorized labels
y_train = np.asarray(train_labels).astype('float32')
y_test = np.asarray(test_labels).astype('float32')

x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]

model = keras.models.Sequential()
model.add(keras.layers.Dense(16, activation='relu', input_shape=(10000,), name='reviews'))
model.add(keras.layers.Dense(16, activation='relu'))
model.add(keras.layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
              loss='binary_crossentropy',
              metrics=['accuracy'])
estimator_model = keras.estimator.model_to_estimator(keras_model=model)

def input_function(features,labels=None,shuffle=False,epochs=None,batch_size=None):
    input_fn = tf.estimator.inputs.numpy_input_fn(
        x={"reviews_input": features},
        y=labels,
        shuffle=shuffle,
        num_epochs=epochs,
        batch_size=batch_size
    )
    return input_fn

estimator_model.train(input_fn=input_function(partial_x_train, partial_y_train, True,20,512))
score = estimator_model.evaluate(input_function(x_val, labels=y_val))
print(score)

Answer 1

You should reshape your labels as 2d-tensor (the first dimension will be the batch dimension and the second the scalar label):

# Our vectorized labels
y_train = np.asarray(train_labels).astype('float32').reshape((-1,1))
y_test = np.asarray(test_labels).astype('float32').reshape((-1,1))

Answer 2

If you're doing a binary classification, make sure that that your last Dense layer has just (None, 1) in its shape, not None, 2)

tf.keras.layers.Dense(1, activation="sigmoid") # binary activation output

Answer 3

Check your network using model.summary()

You eventually need to thin down the network to have the same outputs as your classes. For example, doing OCR for numbers needs and final output of Dense(10) (for numbers 0 to 9).

For example characterizing dogs vs. cats. The final layer has to have two outputs (0-dog, 1-cat)

Answer 4

We can solve this problem by matching the output with the dimension of the label by adding a Flatten layer after the Dense layer:

model.add(Flatten())

or by adding:

model.add(GlobalAveragePooling2D())

See this GitHub issue for full details

Answer 5

If you're doing binary cross-entropy, then your dataset probably has 2 classes and the error is coming because your label vectors (both in testing and training) have the form [0,1,0,1,1,1,0,0,1,...]. To one-hot encode binary labels, the following function can be used: Labels = tf.one_hot(Labels, depth=2)