tf.keras (RNN) Layer issues when running model.fit()

Question

I'm building an RNN to analyze motion capture (MoCap) data using TensorFlow, Pandas, and Keras.

About my data:

• Data is obtained through pandas.read_csv and has a shape of (832, 165)
• Each row denotes a whole frame of data in a movement sequence (832 frames)
• Each column denotes the rotational data for a joint (165 joints total)

I'm attempting to feed the data in one row at a time. The output should be the next frame in the movement sequence. I keep running into different types of errors when running model.fit .

I've attached a series of photos representing the different attempts to make the model work. If someone could provide some guidance as to why it's not working and how to fix I'd greatly appreciate it.

As a side note, each version of my code is different. I'm okay with using any as long as it ends up working, so when providing feedback if you could identify which version of my code you're talking about?

Uses tf.data.Dataset as input

Version 1 Code / Output

Version 2 Code / Output

Version 3: [Code] [Output]

Uses pandas arrays for input and target

Version 4 Code / Output

Version 5 Code / Output

Answer 1

Using Code 4 as a basis for troubleshooting, I noticed that you are passing incompatible shapes to the layers.

This line model.add(keras.layers.InputLayer(input_shape = (N_TIMESTEPS, N_FEATURES))) expects your data to have the same shape. Whereas your data have (832, 165) , which is the N_SAMPLES on the first index and the N_FEATURES , the N_TIMESTEPS is missing .

First, you should create a modified dataset that will generate a shape of (N_SAMPLES, N_TIMESTEPS, N_FEATURES) .

Here is an example to generate a dummy dataset:

data = tf.random.normal((N_SAMPLES, N_TIMESTEPS, N_FEATURES))
target = tf.random.normal((N_SAMPLES, N_TIMESTEPS, N_FEATURES))

The N_TIMESTEPS in your data is important in LSTM as it determines how many TIME_STEPS to consider per update.

Here is the complete code used to simulate successful execution in Google Colab.

%tensorflow_version 2.x  # To ensure latest Tensorflow version in Google Colab

import tensorflow as tf
import tensorflow.keras as keras

print(tf.__version__) # Tensorflow 2.2.0-rc3

BATCH_SIZE = 1
N_TIMESTEPS = 10
#Data is obtained through pandas.read_csv and has a shape of (832, 165)
#Each row denotes a whole frame of data in a movement sequence (832 frames)
#Each column denotes the rotational data for a joint (165 joints total)
# N_SAMPLES = data.values.shape[0]
# N_FEATURES = data.values.shape[1]
N_SAMPLES  = 832
N_FEATURES = 165

def get_compiled_model():
  model = keras.Sequential()
  model.add(keras.layers.InputLayer(input_shape = (N_TIMESTEPS, N_FEATURES)))
  model.add(keras.layers.LSTM(35, activation = 'relu', return_sequences = True))
  model.add(keras.layers.LSTM(35, activation = 'relu', return_sequences = True))
  model.add(keras.layers.Dense(165, activation = 'tanh'))

  model.compile(optimizer = 'adam',
                loss = 'mse',
                metrics = ['accuracy'])

  return model

model = get_compiled_model()
model.summary()

data = tf.random.normal((N_SAMPLES, N_TIMESTEPS, N_FEATURES))
target = tf.random.normal((N_SAMPLES, N_TIMESTEPS, N_FEATURES))

model.fit(data, target, epochs = 15, batch_size = BATCH_SIZE, shuffle = False)

Hope this helps you.

You could read more about the Tensorflow Keras Guide using RNN in this link .