how to convert a tensor to a numpy array without enabling the run_eagerly flag in keras

Question

I am writing my own metric call-back functions where I use sklearn to calculate the metrics and for that I need to have the y_true and y_pred tensors as numpy arrays. My functions look like this:

def precision_macro(y_true, y_pred):
    # get the y_true and y_pred tensors as 1-D numpy array
    y_true_array = np.array(true)
    y_pred_array = np.array(pred)
    ....................
    ....................
        CALCULATIONS
    ....................
    ....................
    precision = precision_score(y_true_array, y_pred_array, average="macro", zero_division=0)
    return precision

Everything works fine if I set the run_eargly=True during the compile call like this:

model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=lr),
                      loss='binary_crossentropy',
                      metrics=model_metrics,
                      run_eagerly=True) 

but this is very costly and way slower than with the flag set to False but if I don't set the flag to True I have a problem with the conversion. Here are things I have tried without setting the run_eagrly flag to True and didn't work:

If I just don't set the run_eagerly flag, I get the following error

NotImplementedError: Cannot convert a symbolic Tensor (ExpandDims:0) to a numpy array. This error may indicate that you're trying to pass a Tensor to a NumPy call, which is not supported

Then I tried

import tensorflow.keras.backend as K

def precision_macro(y_true, y_pred):
    # get the y_true and y_pred tensors as 1-D numpy array
    y_true_array = K.eval(y_true)
    y_pred_array = K.eval(y_pred)
    ....................

Or I try to call the numpy function on the tensors

def precision_macro(y_true, y_pred):
    # get the y_true and y_pred tensors as 1-D numpy array
    y_true_array = y_true.numpy()
    y_pred_array = y_pred.numpy()
    ....................

I also tried run this inside a session like this:

import tensorflow.keras.backend as K
import tensorflow as tf

def precision_macro(y_true, y_pred):
    sess = tf.compat.v1.Session()
    # get the y_true and y_pred tensors as 1-D numpy array
    with sess.as_default():
       y_true_array = K.eval(y_true)
       y_pred_array = K.eval(y_pred)
    ....................

I get the following error for all thress cases

AttributeError: 'Tensor' object has no attribute 'numpy'

I tried to run it like this:

import tensorflow.keras.backend as K
import tensorflow as tf

def precision_macro(y_true, y_pred):
    sess = tf.compat.v1.Session()
    # get the y_true and y_pred tensors as 1-D numpy array
    with sess.as_default():
       y_true_array = sess.run(y_true)
       y_pred_array = sess.run(y_pred)
    ....................

I get the following error

InvalidArgumentError: 2 root error(s) found. (0) Invalid argument: You must feed a value for placeholder tensor 'iterator' with dtype resource (1) Invalid argument: You must feed a value for placeholder tensor 'iterator' with dtype resource

I tried downgrading numpy from 1.19.5 to 1.18.5 but that didn't work either, I get the same errors

trying tf.numpy_function()

import tensorflow as tf

def numpy_function(y_true, y_pred):
    # get the y_true and y_pred tensors as 1-D numpy array
    y_true_array = y_true
    y_pred_array = y_pred
    .... calculations ....
    precision = precision_score(y_true_array, y_pred_array, average="macro", zero_division=0)
    precision_per_class.append(precision_score(y_true_array, y_pred_array, average=None, zero_division=0))
    return precision

the function I pass as metric

@tf.function
def precision_macro(y_true, y_pred):
    score = tf.numpy_function(numpy_function, [y_true, y_pred], tf.float32)

    return score

and I compile like this, where model_metrics = ['accuracy', precision_macro]

model.compile(optimizer=keras.optimizers.RMSprop(learning_rate=lr),
                      loss='binary_crossentropy',
                      metrics=model_metrics)

I get the following error

TypeError: 'Tensor' object does not support item assignment

I am using keras = 2.6. tensorflow = 2.6 numpy = 1.19.5

So, can anybody help me?

Answer 1

You could try tf.numpy_function . It receives a function as argument and will automatically convert tensors to numpy arrays before passing into the function and convert the outputs of the function back to tensors. See example:

def my_numpy_func(y_true, y_pred):
  # y_true and y_pred are already numpy arrays
  # put all numpy or sklearn codes here
  return metrics.mean_squared_error(y_true, y_pred)

@tf.function
def custom_metric(y_true, y_pred):
  # y_true and y_pred are tensors
  # no numpy or sklearn code is allowed here
  score = tf.numpy_function(my_numpy_func, [y_true, y_pred], tf.float32)
  return score

It will allow you to have run_eagerly = False and in theory, run faster as most computations except this custom metric work in graph mode.