Run TensorFlow op in graph mode in tf 2.x

Question

I would like to benchmark some TensorFlow operations (for example between them or against PyTorch). However most of the time I will write something like:

import numpy as np
import tensorflow as tf

tf_device = '/GPU:0'
a = np.random.normal(scale=100, size=shape).astype(np.int64)
b = np.array(7).astype(np.int64)

with tf.device(tf_device):
  a_tf = tf.constant(a)
  b_tf = tf.constant(b)

%timeit tf.math.floormod(a_tf, b_tf)

The problem with this approach is that it does the computation in eager-mode (I think in particular that it has to perform GPU to CPU placement). Eventually, I want to use those ops in a tf.keras model and therefore would like to evaluate their performance in graph mode.

What is the preferred way to do it?

My google searches have led to nothing and I don't know how to use sessions like in tf 1.x.

Answer 1

What you are looking for is tf.function . Check this tutorial and this docs .

As the tutorial says, in TensorFlow 2, eager execution is turned on by default. The user interface is intuitive and flexible (running one-off operations is much easier and faster), but this can come at the expense of performance and deployability. To get performant and portable models, use tf.function to make graphs out of your programs.

Check this code:

import numpy as np
import tensorflow as tf
import timeit

tf_device = '/GPU:0'

shape = [100000]
a = np.random.normal(scale=100, size=shape).astype(np.int64)
b = np.array(7).astype(np.int64)

@tf.function
def experiment(a_tf, b_tf):
  tf.math.floormod(a_tf, b_tf)

with tf.device(tf_device):
  a_tf = tf.constant(a)
  b_tf = tf.constant(b)

# warm up
experiment(a_tf, b_tf)
print("In graph mode:", timeit.timeit(lambda: experiment(a_tf, b_tf), number=10))
print("In eager mode:", timeit.timeit(lambda: tf.math.floormod(a_tf, b_tf), number=10))