打印张量值 tensorflow 2.4

Question

I tied to print one value while i train my Transformer.

@tf.function
def train_step(inp, tar):
    tar_inp = tar[:, :-1]
    tar_real = tar[:, 1:]
    global i
    if i == 1:
      print('__________________')
      tf.print('Inp: ', inp, output_stream=sys.stdout)
      tf.print('Tar: ', tar, output_stream=sys.stdout)
      tf.print('Tar_inp: ', tar_inp, output_stream=sys.stdout)
      tf.print('Tar_real: ', tar_real, output_stream=sys.stdout)
      i += 1
    .......

But tf.print doesn't print anything. However my first print('_') works What did I do wrong? Pls, help me to print my tensors.

UPDATE : U also could explain to me the structure of tar_inp and tar_real instead of fixing tf. print tf. print .

Answer 1

Try changing it to print() with tensor.numpy() -

@tf.function
def train_step(inp, tar):
    tar_inp = tar[:, :-1]
    tar_real = tar[:, 1:]
    global i
    if i == 1:
      print('__________________')
      print('Inp: ', inp.numpy())
      print('Tar: ', tar.numpy())
      print('Tar_inp: ', tar_inp.numpy())
      print('Tar_real: ', tar_real.numpy())
      i += 1

---

W.r.t the tar_inp and tar_real , indexing works the same way on tensors as it does on numpy arrays but it returns a tensor object. So you can index, and then convert pull the values as a numpy array later.

print(tf.convert_to_tensor([1,2,3]).numpy()) #get numpy
print(tf.convert_to_tensor([1,2,3])[1:].numpy()) #get numpy after indexing
print(tf.convert_to_tensor([1,2,3]).numpy()[1:]) #index after getting numpy

[1 2 3]
[2 3]
[2 3]

Answer 2

You can use eval from tf.keras.backend .

Simple example:

import tensorflow as tf
import tf.keras.backend as K

x = tf.constant([3.,3.])
print(x)
print(K.eval(x))

Answer 3

for (batch, (inp, tar)) in enumerate(dataset):
    tar_inp = tar[:, :-1]
    tar_real = tar[:, 1:]
    print(tar_inp, tar_real, sep='\n\n')
    break

[[    2    90 21617 ...     0     0     0]
 [    2   282  1103 ...     0     0     0]
 [    2  1120 20666 ...     0     0     0]
 ...
 [    2   637    19 ...     0     0     0]
 [    2   200  6206 ...     0     0     0]
 [    2   647    19 ...     0     0     0]], shape=(64, 74), dtype=int32)

tf.Tensor(
[[   90 21617   130 ...     0     0     0]
 [  282  1103  1514 ...     0     0     0]
 [ 1120 20666  4508 ...     0     0     0]
 ...
 [  637    19  1789 ...     0     0     0]
 [  200  6206  9124 ...     0     0     0]
 [  647    19  1487 ...     0     0     0]], shape=(64, 74), dtype=int32)

Answer 4

Use:

model.compile(... run_eagerly=True)

Then everything will work fine. You may have to remove @tf.function (let me know if it's the case).

The reason is that Tensorflow is made to be used with edge devices and/or anything outside of Python. When writing Tensorflow code, you're creating a graph that will be executed independent of Python. If you want it to be executed as Python functions, for instance if you want to print the values within Python, you will need to execute eagerly.