numpy vs pytorch precision

Question

I've a numpy matrix a defined as:

>>> a
>>> array([[ 1.920941165 ,  0.9518795607,  1.5358781432],
       [-0.2418292026,  0.0851087409, -0.2760766872],
       [-0.4161812806,  0.7409229185, -0.3248560283],
       [-0.3439163186,  1.4052927665, -1.612850871 ],
       [ 1.5810794171,  1.1820622504,  1.8063415367]])

If I typecast it to float32, it gives:

>>> a.astype(np.float32)
>>> array([[ 1.9209411 ,  0.95187956,  1.5358782 ],
       [-0.2418292 ,  0.08510874, -0.27607667],
       [-0.41618127,  0.7409229 , -0.32485604],
       [-0.34391633,  1.4052927 , -1.6128509 ],
       [ 1.5810794 ,  1.1820623 ,  1.8063415 ]], dtype=float32)

When I convert original a matrix to a tensor, I get:

>>> torch.tensor(a)
>>> tensor([[ 1.9209411650,  0.9518795607,  1.5358781432],
        [-0.2418292026,  0.0851087409, -0.2760766872],
        [-0.4161812806,  0.7409229185, -0.3248560283],
        [-0.3439163186,  1.4052927665, -1.6128508710],
        [ 1.5810794171,  1.1820622504,  1.8063415367]], dtype=torch.float64)

which looks correct as it retains original values from matrix a . But when I convert float32-typecasted matrix to a tensor, I get different floating point numbers.

>>> torch.tensor(a.astype(np.float32))
>>> tensor([[ 1.9209411144,  0.9518795609,  1.5358781815],
        [-0.2418292016,  0.0851087421, -0.2760766745],
        [-0.4161812663,  0.7409229279, -0.3248560429],
        [-0.3439163268,  1.4052927494, -1.6128509045],
        [ 1.5810793638,  1.1820622683,  1.8063415289]])

Why can't the second tensor(tensor of type-casted matrix) be equal to the second matrix(type-casted one) provided above.

Answer 1

float32 has 24fraction bit ( 7.2 decimal point in decimal), what you see after that is not meaningful. ex: 1.920941165 (9 point). this means if you want to retain all points you shoult represent as 64 float. however when you convert to 32, either in numpy or torch they should be same values, it is only printing is different. torch print till the number of floating that you have set, while numpy truncate only till valid points.

for example:

import numpy as np
np.set_printoptions(precision=10)
a = np.array([1.920941165],dtype=np.float32)

array([ 1.9209411 ], dtype=float32)**

t = torch.tensor(a , dtype=torch.float32)

tensor([ 1.9209411144 ])

however if you look into underlying memory of both (one uint32) are the same:

np.ndarray(1, dtype=np.uint32,buffer=a )

array([ 1073078630 ], dtype=uint32)

import ctypes
ptr = ctypes.cast(t.data_ptr(), ctypes.POINTER(ctypes.c_uint32))
ptr[0]

1073078630