How can I compare two 2D NumPy arrays of size 40k × 40k?

Question

I must compare 2 NumPy arrays of floats. Unfortunately of the order of 10^-3 or smaller, of size 40k × 40k. They must be identical. Can I directly print/mask the elements that differ?

I tried allclose and isclose :

>>> import numpy as np
>>> F_sp = np.load('./Matrix/Fmatrix.npy')
>>> F_org = np.load('./../Org/FMatrix.npy')
>>> print(" all close? {} ".format(np.allclose(F_sp,F_org,equal_nan=True)))
all close? False

>>> print(" is close? {} ".format(np.isclose(F_sp,F_org,equal_nan=True)))
is close? [[ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 ...
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]]

>>> print(" is close? {} ".format(np.isclose(F_sp,F_org,equal_nan=False)))
is close? [[ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 ...
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]
 [ True  True  True ...  True  True  True]]

>>> np.set_printoptions(threshold=sys.maxsize)
>>> print(" is close? {} ".format(np.isclose(F_sp,F_org,equal_nan=True)))
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
  print(" is close? {} ".format(np.isclose(F_sp,F_org,equal_nan=True)))
OverflowError: cannot serialize a string larger than 4GiB

Answer 1

If they must be identical, it is not necessary to check if the values are close, but simply if they are identical.

Using:

identical = F_sp == F_org

should give an array with true and false. To print the elements that differ, use np.where(condition) . This will give an array with the elements that fill the condition.