Tensorflow: Can't extract the feature vector from the penultimate layer

Question

I want to extract the feature vector from an image with a pre-trained network from the penultimate layer.

When I ran:

from neural_network import NET


x = tf.placeholder(tf.float32, shape=[1, 144, 144, 3])

net = NET({'data': x})
sess = tf.InteractiveSession()
sess.run(tf.initialize_all_variables())
net.load('inference.npy', sess)

feature = sess.graph.get_tensor_by_name('fc7:0')

I received the error message:

KeyError: "The name 'fc7:0' refers to a Tensor which does not exist. The operation, 'fc7', does not exist in the graph."

On the other hand, if I replace the last line with:

 feature = sess.graph.get_tensor_by_name('global_pool:0')

then it works.

The end of my neural_network.py file is:

    (self.feed('inception_3b_pool',
               'inception_3b_3x3',
               'inception_3b_double_3x3_2')
         .concat(3, name='inception_3b_output')
         .avg_pool(9, 4, 1, 1, padding='VALID', name='global_pool') 
         .fc(256, relu=False, name='fc7'))

and the definition of the fc layer is:

def fc(self, input, num_out, name, relu=True):
    with tf.variable_scope(name) as scope:
        input_shape = input.get_shape()
        if input_shape.ndims == 4:
            # The input is spatial. Vectorize it first.
            dim = 1
            for d in input_shape[1:].as_list():
                dim *= d
            feed_in = tf.reshape(input, [-1, dim])
        else:
            feed_in, dim = (input, input_shape[-1].value)
        weights = self.make_var('weights', shape=[dim, num_out])
        biases = self.make_var('biases', [num_out])
        op = tf.nn.relu_layer if relu else tf.nn.xw_plus_b
        fc = op(feed_in, weights, biases, name=scope.name)
        return fc

Answer 1

Usually output of fc layers is a BiasAdd node and output tensor name is something like fc7/BiasAdd:0 . Though it depends on the implementation of your fc method.

I think it makes sense to load your graph in tensorboard and look for output node name.

Answer 2

feature = sess.graph.get_tensor_by_name("fc7/fc7:0") 

works

see also https://www.tensorflow.org/versions/r0.11/how_tos/variable_scope/index.html

Answer 3

Do

[n.name for n in tf.get_default_graph().as_graph_def().node]

to list all the tensor names of the graph. Print the output of required tensor using sess.graph.get_tensor_by_name(<tensor_operation_name>:<output_index>") .

Ex: "fc7/fc7:0"