在 jupyter 中的 `np.ma.array` 问题中屏蔽了 `np.nan`

Question

Let's run in the Anaconda Jupyter the Python3 NumPy code:

y = np.ma.array(np.matrix([[np.nan, 2.0]]), mask=[0, 1])
m = (y < 0.01)

and we have the warning: /.../anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in less .

Substituting np.nan with 1.0 etc. --- no warning.

Why the np.nan can not be masked and then compared?

Answer 1

MA has several strategies to implementing methods.

1) evaluate the method on y.data , and make a new ma with y.mask . It may suppress any runtime warnings.

2) evaluate the method on y.filled() # with the default fill value

3) evaluate the method on y.filled(1) # or some other innocuous value

4) evaluate the method on y.compressed()

5) evaluate the method on y.data[~y.mask]

multiplication, for example use filled(1) , and addition uses filled(0) .

It appears that the comparisons are done with 1).

I haven't studied the ma code in detail, but I don't think it does 5).

If you are using ma just to avoid the runtime warning, there are some alternatives.

• there's a collection of np.nan... functions that filter out nan before calculating

• there are ways of surpressing runtime warnings

• ufuncs have a where parameter that can be used to skip some elements. Use it with an out parameter to define the skipped ones.

===

Looking a np.ma.core.py I see functions like ma.less .

In [857]: y = np.ma.array([np.nan, 0.0, 2.0], mask=[1, 0, 0])                                  
In [858]: y >1.0                                                                               
/usr/local/bin/ipython3:1: RuntimeWarning: invalid value encountered in greater
  #!/usr/bin/python3
Out[858]: 
masked_array(data=[--, False, True],
             mask=[ True, False, False],
       fill_value=True)
In [859]: np.ma.greater(y,1.0)                                                                 
Out[859]: 
masked_array(data=[--, False, True],
             mask=[ True, False, False],
       fill_value=True)

Looking at the code, ma.less and such are a MaskedBinaryOperation class, and use 1) - evaluate on the data with

np.seterr(divide='ignore', invalid='ignore')

The result mask is logical combination of the arguments' masks.

https://docs.scipy.org/doc/numpy/reference/maskedarray.generic.html#operations-on-masked-arrays

Answer 2

Making the issue more simple, let's assume:

y = np.ma.array([np.nan, 0.0, 2.0], mask=[1, 0, 0])
m = (y > 1.0)
print(y, y.shape) ; print(y[m], y[m].shape, m.shape)

and the output is:

[-- 0.0 2.0] (3,)
[2.0] (1,) (3,)

with the RuntimeWarning: /.../anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:2: RuntimeWarning: invalid value encountered in greater .

Changing:

...
m = (y != 2.0)
...

We get:

[-- 0.0 2.0] (3,)
[-- 0.0] (2,) (3,)

so we have a masked element and the result without any RuntimeWarning.

Changing now:

...
m = y.mask.copy() ; y[np.isnan(y)] = 9.0 ; y.mask = m ; m = (y > 1.0)
...

We get (without RuntimeWorning):

[-- 0.0 2.0] (3,)
[-- 2.0] (2,) (3,)

This work-around is however strange (by setting arbitrary value in the place of np.nan and mask saving). Comparing something with masked should be always masked , shouldn't it?