Tensorflow: Understanding the layer structure of LSTM model

Question

I'm new to tensorflow and LSTM and I'm having some trouble understanding the shape and structure of the network (weights, biases, shape of inputs and logs).

In this specific piece of code taken from here

def recurrent_neural_network(x):
    layer = {'weights':tf.Variable(tf.random_normal([rnn_size,n_classes])),
         'biases':tf.Variable(tf.random_normal([n_classes]))}

    x = tf.transpose(x, [1,0,2])
    x = tf.reshape(x, [-1, chunk_size])
    x = tf.split(x, n_chunks, 0)

    lstm_cell = rnn_cell.BasicLSTMCell(rnn_size,state_is_tuple=True)
    outputs, states = rnn.static_rnn(lstm_cell, x, dtype=tf.float32)

    output = tf.matmul(outputs[-1],layer['weights']) + layer['biases'])

    return output

1. Can someone please explain why we need to convert x to this specific format (transpose -> reshape -> split)

2. Why weights are defined as [rnn_size, n_classes] and biases defined as [n_classes].

3. What is the exact structure of the network that is being formed and how are the weights connected, I don't understand that properly.

4. Is there any site or reference that I could read that would help?

Thanks.

Answer 1

For the general network structure, LSTMs are an extension of RNN networks. For explanation of RNN network structure, take a look at this classic blog post

For the actual LSTMs, try this post (which also has an RNN explanation)

These are not very formal, but they should be much easier to read and understand than academic papers.

Once you read these, the rest should not be very hard. The reason for the transformations of X is because that is the format static_rnn expects. And rnn_size is the size of the LSTM cell, so thats why the weights are shaped that way.