为什么 tanh function 在 tensorflow 和 pytorch 中返回不同？

Question

I find that tensorflow and pytorch tanh result is different, I want to know why did this happen? I know that the difference is very small, so is this acceptable?

import numpy as np
import tensorflow as tf
import torch

np.random.seed(123)
tf.random.set_seed(123)
torch.manual_seed(123)

batch, sentence_length, embedding_dim = 20, 5, 10
value = np.random.random((batch, sentence_length, embedding_dim)).astype("f")
value = value * 10

tf_x = tf.constant(value, dtype=tf.float32)
tf_out = tf.math.tanh(tf_x)

pt_x = torch.from_numpy(value)
pt_out = torch.tanh(pt_x)

print((tf_out.numpy() == pt_out.numpy()).all()) # return False
print(((tf_out.numpy() - pt_out.numpy()) < 1e-6).all()) # return True

• tensorflow == 2.5.0
• torch == 1.9.0

Answer 1

Running your code with the following line at the end:

print(np.allclose(tf_out.numpy(), pt_out.numpy()))  # Returns True

You will receive True . I do not know exactly how tensorflow and pytorch compute the tanh oppeartion, but when working with floating points, you rarely are exactely equal. However, you should be receiving equal results up to a certain tolerance, which is exactly what np.allclose() checks. Read more onallclose here