Tensorflow RNN output tensor shapes

Question

I'm quite new to tensorflow and I can't quite get how to shape tensors so that I would get the output as a single number. Basically, my recurrent network should guess the next number. Instead, with each prediction it returns me a list with five numbers? I guess either one or more of my tensors are misshaped.

My input data is formatted to be around 2000 lists with 5 features each like this:

[
  np.array ([
              [1],[2],[3],[4],[5]
            ]) 
]

This is the code for the RNN:

cell_units = 400
batch_size = 5
no_of_epochs = 500

data = tf.placeholder (tf.float32, [None, 5, 1])
target = tf.placeholder (tf.float32, [None, 1, 1])


weight = tf.Variable (tf.random_normal ([cell_units, 5, 1]))
bias = tf.Variable (tf.random_normal([1, 1]))


cell = tf.contrib.rnn.BasicRNNCell (num_units = cell_units)

output, states = tf.nn.dynamic_rnn (cell, data, dtype=tf.float32)


output = tf.transpose (output, [1, 0, 2])


activation = tf.matmul (output, weight) + bias


cost = tf.reduce_mean (
                        (
                            tf.log (tf.square (activation - target))
                        )
                      )

optimizer = tf.train.AdamOptimizer (learning_rate = 0.01).minimize(cost)

with tf.Session () as sess:

    sess.run (tf.global_variables_initializer ())
    no_of_batches = int (len (train_x) / batch_size)

    for i in range(no_of_epochs):
        start = 0
        for j in range(no_of_batches):
            inp = train_x [start:start+batch_size]
            out = train_y [start:start+batch_size]
            start += batch_size

            sess.run (optimizer, {data: inp, target: out})

Answer 1

tf.nn.dynamic_rnn expects inputs of shape [batch_size, max_time, ...] . In your example batch_size is dynamic (ie, unknown) and max_time is 5 (ie, number of time steps.). Naturally RNN's output contains 5 entries, one per input step: [None, 5, cell_units] .

As @ Ishant Mrinal suggested you can select the last output step.

weight = tf.Variable (tf.random_normal ([cell_units, 1]))
bias = tf.Variable (tf.random_normal([1, 1]))

cell = tf.contrib.rnn.BasicRNNCell (num_units = cell_units)
output, states = tf.nn.dynamic_rnn (cell, data, dtype=tf.float32)
# Get the last step (4th index).
output = tf.squeeze(tf.transpose (output, [0, 2, 1])[:,:,4]) # Shape of [batch_size, cell_units].
activation = tf.matmul (output, weight) + bias

activation has shape of [batch_size, 1] .