如何为Tensorflow估算器输入正确的形状？

Question

I am trying to build a Tensorflow estimator to use on SageMaker . The main function trains and evaluates the estimator. Despite my best attempts, I keep getting the following error:

ValueError: Input 0 of layer inputs is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [50, 41]

def keras_model_fn(hyperparameters):
    """keras_model_fn receives hyperparameters from the training job and returns a compiled keras model.
    The model will be transformed into a TensorFlow Estimator before training and it will be saved in a 
    TensorFlow Serving SavedModel at the end of training.

    Args:
        hyperparameters: The hyperparameters passed to the SageMaker TrainingJob that runs your TensorFlow 
                         training script.
    Returns: A compiled Keras model
    """
    model = tf.keras.models.Sequential()
    model.add(tf.keras.layers.LSTM(32, name='inputs', input_shape=( None, 41)))
    model.add(tf.keras.layers.Dense(11, activation='softmax', name='dense'))
    model.compile(loss='categorical_crossentropy',
                  optimizer='rmsprop',
                  metrics=['accuracy'])

    return model


def train_input_fn(training_dir=None, hyperparameters=None):
    # invokes _input_fn with training dataset
    dataset = tf.data.Dataset.from_tensors(({INPUT_TENSOR_NAME: x_train}, y_train))
    dataset = dataset.repeat()
    return dataset.make_one_shot_iterator().get_next()

def eval_input_fn(training_dir=None, hyperparameters=None):
    # invokes _input_fn with evaluation dataset

    dataset =  tf.data.Dataset.from_tensors(({INPUT_TENSOR_NAME: x_test}, y_test))
    return dataset.make_one_shot_iterator().get_next()

if __name__ == '__main__':
    print(x_train.shape, y_train.shape)
    tf.logging.set_verbosity(tf.logging.INFO)
    model = keras_model_fn(0)
    estimator = tf.keras.estimator.model_to_estimator(keras_model=model)
    train_spec = tf.estimator.TrainSpec(input_fn=train_input_fn, max_steps=1000)
    eval_spec = tf.estimator.EvalSpec(input_fn=eval_input_fn)
    tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

My inputs and output shapes are:

(52388, 50, 41) (52388, 11)

Answer 1

from_tensors convert the input tensor to single big Tensor. For example, if you run the below example:

import tensorflow as tf

tf.enable_eager_execution()

dataset2 = tf.data.Dataset.from_tensors(
    (tf.random_uniform([52388, 50, 41], maxval=10, dtype=tf.int32),
     tf.random_uniform([52388, 11], maxval=10, dtype=tf.int32)))

for i, item in enumerate(dataset2):
    print('element: ' + str(i), item[0], item[1])

You notice, we only iterate dataset once while we expect to iterate it 52388 times!

Now assume we want to feed this single big tensor to our model. Tensorflow converts to [None, 1] which None is our batch size. On the other hand, you specify the input of the model with [None, 41] which means the model expect an input with shape [None, None, 41] . Hence, this inconsistency causes the error.

How to fix it?

Use from_tensor_slices .

Still giving me dimension error, how to fix it? Define the input dimension for LSTM:

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.LSTM(32, name='inputs', input_shape=(50, 41)))
model.add(tf.keras.layers.Dense(11, activation='softmax', name='dense'))
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])
model.compile(loss='categorical_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])