如何在Tensorflow中获得LSTM的密集层输出？

Question

I am using Tensorflow for modelling an LSTM with a single dense layer. What I would like to accomplish is to obtain the dense layer output/hidden representations from the LSTM. I have checked that similar methodology is available in Keras, but how about doing it in Tensorflow? I append my code below which is specific to the problem (referring to LSTM on sequential data, predicting a discrete column ) :-

# clear graph (if any) before running
tf.reset_default_graph()

X = tf.placeholder(tf.float32, [None, time_steps, inputs], name = "Inputs")
y = tf.placeholder(tf.float32, [None, outputs], name = "Outputs")

# LSTM Cell
cell = tf.contrib.rnn.BasicLSTMCell(num_units=neurons, activation=tf.nn.relu)
cell_outputs, states = tf.nn.dynamic_rnn(cell, X, dtype=tf.float32)

# pass into Dense layer
stacked_outputs = tf.reshape(cell_outputs, [-1, neurons])
out = tf.layers.dense(inputs=stacked_outputs, units=outputs)

# squared error loss or cost function for linear regression
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
    labels=y, logits=out))

# optimizer to minimize cost
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
training_op = optimizer.minimize(loss)


with tf.Session() as sess:
    # initialize all variables
    tf.global_variables_initializer().run()
    tf.local_variables_initializer().run()

    # Train the model
    for steps in range(epochs):
        mini_batch = zip(range(0, length, batch_size),
                         range(batch_size, length + 1, batch_size))

        # train data in mini-batches
        for (start, end) in mini_batch:
            sess.run(training_op, feed_dict={X: X_train[start:end, :, :],
                                             y: y_train[start:end, :]})

        # print training performance
        if (steps + 1) % display == 0:
            # evaluate loss function on training set
            loss_fn = loss.eval(feed_dict={X: X_train, y: y_train})
            print('Step: {}  \tTraining loss: {}'.format((steps + 1), loss_fn))

The code I have appended is specific to the training set, but I believe the process should be quite similar to feed in a dictionary for the test set. Is there any one liner/short code segment which can return the dense layer output (hidden representations of input data). Any help in this regard is highly appreciated.

Answer 1

Yes, like this:

_, d_out = sess.run([training_op, out], feed_dict={X: X_train[start:end, :, :],
                                         y: y_train[start:end, :]})
print(d_out)

Answer 2

When you are inside a Session context manager, this is the shortest way: out_vals = out.eval({X: X_train})

It is equivalent to this: out_vals = sess.run(out, feed_dict={X: X_train})

You don't need to feed labels for forward propagation (if you just evaluating the dense layer).