Keras model.fit() giving out TypeError: 'NoneType' object is not callable

Question

I am trying to experiment with Keras to implement Siamese Network to implement one-shot face recognition model. But I am running into an error incomprehensible to me, some help is requested.

The model which I am using is an encoder model which takes in (299,299,3) images (anchor image, positive image as well as a negative image) and outputs the 1000 dimensional encoded vector of each image. This is similar to the InceptionV3 model with the classification head. I am also using a custom triplet loss function for the same. My model is as below:

class SiameseNet(tf.keras.layers.Layer):

  def __init__(self, model):
    self.model = model  # This is the image feature extraction model (similar to InceptionV3)
    super().__init__()

  def call(self, feat):
    feats = self.model(feat[0])
    nfeats = self.model(feat[1])

    return [feats, nfeats]

and the loss function is as below

def triplet_loss(y_true, y_pred, alpha=1e-2):
  return max(tf.reduce_sum((y_pred[0]-y_true)**2 - (y_pred[0]-y_pred[1])**2) + alpha, 0)

There are three arrays named images (anchor images) and negatives (negative images) both with shape (500,299,299,3) (where 500 is the number of training examples) and positives (positive image features) with shape (500,1000) . All these are numpy arrays.

My model code is as shown below

image_input = tf.keras.layers.Input(shape=(299,299,3), name='image_input')
negative_input = tf.keras.layers.Input(shape=(299,299,3), name='negative_input')

siamese = SiameseNet(image_features_extract_model)([image_input, negative_input])

model = tf.keras.Model(inputs=[image_input, negative_input], outputs=siamese)

model.compile(optimizer=tf.keras.optimizers.Adam(), loss=triplet_loss, metrics=['accuracy'])

The compilation works fine with the output

Model: "functional_3"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
image_input (InputLayer)        [(None, 299, 299, 3) 0                                            
__________________________________________________________________________________________________
negative_input (InputLayer)     [(None, 299, 299, 3) 0                                            
__________________________________________________________________________________________________
siamese_net (SiameseNet)        [(None, 1000), (None 23851784    image_input[0][0]                
                                                                 negative_input[0][0]             
==================================================================================================
Total params: 23,851,784
Trainable params: 23,817,352
Non-trainable params: 34,432

but while running

model.fit([images, negatives], positives, epochs=10, batch_size=8, verbose=2)

I get the following error which I require help with

Epoch 1/10
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-16-83443f79d005> in <module>()
----> 1 model.fit([images, negatives], positives, epochs=10, batch_size=8, verbose=2)
      2 # model.fit(train, epochs=10, verbose=2)

3 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in _method_wrapper(self, *args, **kwargs)
    106   def _method_wrapper(self, *args, **kwargs):
    107     if not self._in_multi_worker_mode():  # pylint: disable=protected-access
--> 108       return method(self, *args, **kwargs)
    109 
    110     # Running inside `run_distribute_coordinator` already.

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)
   1096                 batch_size=batch_size):
   1097               callbacks.on_train_batch_begin(step)
-> 1098               tmp_logs = train_function(iterator)
   1099               if data_handler.should_sync:
   1100                 context.async_wait()

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in __call__(self, *args, **kwds)
    778       else:
    779         compiler = "nonXla"
--> 780         result = self._call(*args, **kwds)
    781 
    782       new_tracing_count = self._get_tracing_count()

/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py in _call(self, *args, **kwds)
    805       # In this case we have created variables on the first call, so we run the
    806       # defunned version which is guaranteed to never create variables.
--> 807       return self._stateless_fn(*args, **kwds)  # pylint: disable=not-callable
    808     elif self._stateful_fn is not None:
    809       # Release the lock early so that multiple threads can perform the call

TypeError: 'NoneType' object is not callable

I am running the code on Google Colab on a CPU.
Please help me out with this problem. Thanks.

Answer 1

I would like to answer my own question here because the problem which I was facing was completely different and have been figured out later. The problem specific to my situation was caused because of the triplet_loss function. The loss function expected computations in terms of Tensors while the code did computations in terms of Numpy Arrays. Changing this fixed the error for me and the code works great now.

The function implementation should have been

def triplet_loss(y_true, y_pred, alpha=0.2):
  return tf.maximum(tf.reduce_sum((y_pred[0]-y_true)**2) - tf.reduce_sum((y_pred[0]-y_pred[1])**2)) + tf.constant(alpha), tf.constant(0.0))

and it worked for my case. No other changes were required.

PS: Here, alpha needs to be a bigger number than my previously chosen value of 1e-2 and the learning rate needs to be reduced to 1e-5 by using

optimizer = tf.keras.optimizers.Adam(lr=1e-5)

in model.compile() .

Answer 2

I don't know if it's exactly what you're looking for, but I can get a model with similar shape to work as follows.

The main differences are:

1. return np.stack([feats,nfeats]) .
2. The whole model is a Sequential model, so I'm not depending on keras to build anything based on Input tensors (except for the first layer).
3. The input to the SiameseNet is the output of a convolutional layer, so I slice it to get the right shape.
4. I use a simple MLP for the SiameseNet , but this is just to make it easier to play with.

class SiameseNet(tf.keras.layers.Layer):

  def __init__(self, model):
    super().__init__()
    self.model = model  # This is the image feature extraction model (similar to InceptionV3)

  def call(self, feat):
    # I'm slicing the input this way because I'm expecting input from some convolutional layer
    feats = self.model(feat[:,:,0])
    nfeats = self.model(feat[:,:,1])

    # keras want this output insted of [feats, nfeats]
    return tf.stack([feats, nfeats])


def triplet_loss(y_true, y_pred, alpha=1e-2):
  return max(tf.reduce_sum((y_pred[0]-y_true)**2 - (y_pred[0]-y_pred[1])**2) + alpha, 0)


if __name__ == '__main__':
    # basic MLP for the siamese model
    mlp = tf.keras.models.Sequential()
    mlp.add(tf.keras.Input(shape=(10)))
    mlp.add(tf.keras.layers.Dense(units=10, activation='relu'))

    siamese = SiameseNet(mlp)

    # the "entire model"
    model = tf.keras.Sequential()
    # convolution is used here purely for convenience, to split an input into two channels
    model.add(tf.keras.layers.Conv1D(2,1))
    model.add(siamese)

    # your stuff
    model.compile(optimizer=tf.keras.optimizers.Adam(), loss=triplet_loss, metrics=['accuracy'])
    model.build(input_shape=[1,10,1])
    model.summary()