Combine tensorflow low level API (tensors/placeholders) with Keras model

Question

According to tensorflow . Using tf.keras.Input gives a placeholder and using tf.keras.layers.Dense gives a Tensor. So I wanted to test the equivalence using Tensors and Placeholders with tensorflow low level API's and then use the keras high level API to train my model. This is my code:

tf.reset_default_graph()  

inputs = tf.placeholder(tf.float32, shape=[None, 32])
W_h = tf.get_variable(name="W_h", shape=[32, 64], initializer=tf.truncated_normal_initializer(stddev=0.01))
W_out = tf.get_variable(name="W_out", shape=[64, 10], initializer=tf.truncated_normal_initializer(stddev=0.01))

h = tf.nn.relu(tf.matmul(inputs, W_h, name="MatMul"), name='relu')
predictions = tf.nn.relu(tf.matmul(h, W_out, name="MatMul"), name='relu')

model = tf.keras.Model(inputs=inputs, outputs=predictions)
model.compile(loss='mean_squared_error', optimizer='sgd') # sgd stands for stochastic gradient descent
model.fit(x_train, y_train, batch_size=32, epochs=5)

However, upon call to tf.keras.Model I get the error:

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,32]
     [[{{node Placeholder}}]]

I do give the input a placeholder right?

---

PS: full error message:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-15-27f92d1f784d> in <module>
      8 predictions = tf.nn.relu(tf.matmul(h, W_out, name="MatMul"), name='relu')
      9 
---> 10 model = tf.keras.Model(inputs=inputs, outputs=predictions)
     11 model.compile(loss='mean_squared_error', optimizer='sgd') # sgd stands for stochastic gradient descent
     12 model.fit(x_train, y_train, batch_size=32, epochs=5)

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\engine\training.py in __init__(self, *args, **kwargs)
    127 
    128   def __init__(self, *args, **kwargs):
--> 129     super(Model, self).__init__(*args, **kwargs)
    130     # initializing _distribution_strategy here since it is possible to call
    131     # predict on a model without compiling it.

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\engine\network.py in __init__(self, *args, **kwargs)
    160         'inputs' in kwargs and 'outputs' in kwargs):
    161       # Graph network
--> 162       self._init_graph_network(*args, **kwargs)
    163     else:
    164       # Subclassed network

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\training\tracking\base.py in _method_wrapper(self, *args, **kwargs)
    455     self._self_setattr_tracking = False  # pylint: disable=protected-access
    456     try:
--> 457       result = method(self, *args, **kwargs)
    458     finally:
    459       self._self_setattr_tracking = previous_value  # pylint: disable=protected-access

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\engine\network.py in _init_graph_network(self, inputs, outputs, name, **kwargs)
    267 
    268     if any(not hasattr(tensor, '_keras_history') for tensor in self.outputs):
--> 269       base_layer_utils.create_keras_history(self._nested_outputs)
    270 
    271     self._base_init(name=name, **kwargs)

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\engine\base_layer_utils.py in create_keras_history(tensors)
    198     keras_tensors: The Tensors found that came from a Keras Layer.
    199   """
--> 200   _, created_layers = _create_keras_history_helper(tensors, set(), [])
    201   return created_layers
    202 

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\engine\base_layer_utils.py in _create_keras_history_helper(tensors, processed_ops, created_layers)
    242             constants[i] = op_input
    243           else:
--> 244             constants[i] = backend.function([], op_input)([])
    245       processed_ops, created_layers = _create_keras_history_helper(
    246           layer_inputs, processed_ops, created_layers)

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\keras\backend.py in __call__(self, inputs)
   3290 
   3291     fetched = self._callable_fn(*array_vals,
-> 3292                                 run_metadata=self.run_metadata)
   3293     self._call_fetch_callbacks(fetched[-len(self._fetches):])
   3294     output_structure = nest.pack_sequence_as(

~\.conda\envs\tensorflow_cpu\lib\site-packages\tensorflow\python\client\session.py in __call__(self, *args, **kwargs)
   1456         ret = tf_session.TF_SessionRunCallable(self._session._session,
   1457                                                self._handle, args,
-> 1458                                                run_metadata_ptr)
   1459         if run_metadata:
   1460           proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder' with dtype float and shape [?,32]
     [[{{node Placeholder}}]]

Answer 1

You can't mix tf.placeholder and tf.keras.Input . In other words, if you want to use the tf.keras API then use tf.keras.Input , or if you want to use the tf native API the go with tf.placeholder . In addition, your choice will reflect other parts of your code. Assuming that you want to go with the tf.keras API, the approach you should partake is the following:

class CustomLayer(tf.keras.layers.Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(CustomLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        # Create a trainable weight variable for this layer.
        self.kernel = self.add_weight(name='kernel', 
                                      shape=tf.TensorShape((input_shape[1], self.output_dim)),
                                      initializer=tf.truncated_normal_initializer(stddev=0.01),
                                      trainable=True)
        super(CustomLayer, self).build(input_shape)  # Be sure to call this at the end

    def call(self, x):
        return tf.nn.relu(tf.matmul(x, self.kernel))

    def compute_output_shape(self, input_shape):
        return (input_shape[0], self.output_dim)

inputs = tf.keras.Input(shape=(32,), dtype=tf.float32)
h = CustomLayer(output_dim=64)(inputs)
predictions = CustomLayer(output_dim=10)(h)
model = tf.keras.Model(inputs=inputs, outputs=predictions)
model.compile(loss='mean_squared_error', optimizer='sgd') # sgd stands for stochastic gradient descent
model.summary()
model.fit(np.random.rand(100,32), np.random.rand(100,10), batch_size=32, epochs=5)

Please note that this approach is relevant if we consider that you use TF 1.14 as you mentioned in the comments. On TF 2.0, this is less complicated and more intuitive. On the other hand, if you want to stick to the TF 1.14 native API, and use tf.placeholder , then you should build the graph using tf.placeholder as nodes that you will use to feed the data. Moreover, regarding your question whether tf.keras.Input returns a placeholder - It does return a placeholder node, that you can use to feed your data. But it does not return a tf.placeholder . The usage of tf.placeholder is the following:

X = tf.placeholder(shape=None, dtype=tf.int32)
Y = tf.placeholder(shape=None, dtype=tf.int32)
add = X + Y
with tf.Session() as sess:
    print(sess.run(add, feed_dict={X: 2, Y: 3}))
    # 5
    print(sess.run(add, feed_dict={X: 10, Y: 9}))
    # 19

As you can see, a static graph is created, and then its nodes are executed with a tf.Session while feeding data in the graph using the tf.placeholder . On the other hand, tf.keras.Input serves the same purpose (That is why in the documentation is called a placeholder), but its use case is relevant with the tf.keras API and not with the tf native API.