Custom loss function Tensorflow 2 [Keras symbolic inputs/outputs do not implement]

Question

I'm trying to use custom loss function and started with simple MSE. Do not pay attention to oscillator function, it needs just for creating data.

import numpy as np
import matplotlib.pyplot as plt

from keras.layers import Input, Dense
from keras.models import Model
import tensorflow as tf


def oscillator(d_, w0_, x):
    assert d_ < w0_
    w = np.sqrt(w0_**2 - d_**2)
    phi = np.arctan(-d_/w)
    A = 1/(2*np.cos(phi))
    cos = np.cos(phi+w*x)
    sin = np.sin(phi+w*x)
    exp = np.exp(-d_*x)
    return exp*2*A*cos



# PARAMETERS:
np.random.seed(5)
N = 20
epochs = 2000
d, w0 = 2, 20
nn_dim = 64


# DATA:
x = np.linspace(0,1,100)
y = oscillator(d,w0,x)

x_train = np.sort(np.random.uniform(0,0.35,N)[:,np.newaxis], axis=0)
y_train = oscillator(d,w0,x_train)

tf_y = tf.Variable(y_train,dtype=tf.float32)



# LAYERS:
input_layer = Input(shape=(1,))
Layer_1 = Dense(nn_dim, activation="tanh")(input_layer)
Layer_2 = Dense(nn_dim, activation="tanh")(Layer_1)
output_layer = Dense(1)(Layer_2)
model = Model(inputs=input_layer, outputs=output_layer)



loss_func = tf.reduce_mean(tf.math.squared_difference(tf_y,output_layer))
model.compile(optimizer='adam', loss=loss_func, metrics=['mse'])


md = model.fit(x_train,y_train,epochs=epochs,verbose=1)
y_pred = model.predict(x[:,np.newaxis])


# PLOTTING:
fig = plt.figure()
plt.plot(md.history['loss'], label='training')
plt.legend()

plt.figure()
plt.plot(x,y,label="Exact solution")
plt.scatter(x_train,y_train,label="Data",color="orange")
plt.plot(x,y_pred,label="Prediction",linestyle="--",color="red")
plt.legend()
plt.show()

The code above produces the following error: TypeError: Keras symbolic inputs/outputs do not implement __len__ . You may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model. This error will also get raised if you try asserting a symbolic input/output directly. Process finished with exit code 1

The problem is in loss_func = tf.reduce_mean(tf.math.squared_difference(tf_y,output_layer)) . I think this is because of different dimensions of tf_y and output_layer . Any ideas how to compute MSE by hand using output_layer and y ?

Answer 1

I've personally never seen a loss defined like that (and I hardly think that it might work), you usually want to create a function:

def loss_func(tf_y, output_layer):
  return tf.reduce_mean(tf.math.squared_difference(tf_y,output_layer))

from the documentation:

loss argument of compile : may be a string (name of loss function), or a tf.keras.losses.Loss instance. See tf.keras.losses. A loss function is any callable with the signature loss = fn(y_true,y_pred), where y_true are the ground truth values, and y_pred are the model's predictions