Interpolating np.nan values in scipy

Question

I want to fill the np.nan data value by interpolating the elements shown in the bold. They are the elements corresponding to same position of np.nan in other dimensions.

import numpy as np
from scipy.interpolate import interp1d
data = np.array([[[3, 2, 1, 3, 2],
                  [**np.nan**, 1, 1, 4, 4],
                  [4, 2, 3, 3, 4],
                  [1, 1, 4, 1, 5],
                  [2, 4, 5, 2, 1]],

                 [[6, 7, 10, 6, 6],
                  [**5**, 9, 8, 6, 9],
                  [6, 10, 9, 8, 10],
                  [6, 8, 7, 10, 8],
                  [10, 9, 9, 10, 8]],

                 [[12, 14, 12, 15, 15],
                  [**21**, 11, 14, 14, 11],
                  [13, 13, 16, 15, 11],
                  [14, 15, 14, 16, 14],
                  [13, 15, 11, 11, 14]]])

result = interp1d(data, kind='cubic')
print result

It results

TypeError: __init__() takes at least 3 arguments (3 given)

What is the best way of doing it? Since I have to process very large array, I am looking for effecient way of doing it.Thanks.

Answer 1

Your questions are just about too board, the interpolation requires no information from the 5*5 matrix, just the values in the same cell in other dimension? If that being the case, well, it is still too board as there are too many interpolation tools to suit different needs. I will say probably the nearest-neighbor method should give you a good start, although the documentation of scipy.interpolate is sort of weak for some:

In [1]:

data = np.array([[[3, 2, 1, 3, 2],
                  [np.nan, 1, 1, 4, 4],
                  [4, 2, 3, 3, 4],
                  [1, 1, 4, 1, 5],
                  [2, 4, 5, 2, 1]],

                 [[6, 7, 10, 6, 6],
                  [5, 9, 8, 6, 9],
                  [6, 10, 9, 8, 10],
                  [6, 8, 7, 10, 8],
                  [10, 9, 9, 10, 8]],

                 [[12, 14, 12, 15, 15],
                  [21, 11, 14, 14, 11],
                  [13, 13, 16, 15, 11],
                  [14, 15, 14, 16, 14],
                  [13, 15, 11, 11, 14]]])
In [2]:

data1=data.reshape((3,-1))
In [3]:
#the one you want to interpolate
data1[:,(np.isnan(data.reshape((3,-1))).any(0))]
Out[3]:
array([[ nan],
       [  5.],
       [ 21.]])
In [4]:
#the other 'good' data points
data1[:,~(np.isnan(data.reshape((3,-1))).any(0))]
Out[4]:
array([[  3.,   2.,   1.,   3.,   2.,   1.,   1.,   4.,   4.,   4.,   2.,
          3.,   3.,   4.,   1.,   1.,   4.,   1.,   5.,   2.,   4.,   5.,
          2.,   1.],
       [  6.,   7.,  10.,   6.,   6.,   9.,   8.,   6.,   9.,   6.,  10.,
          9.,   8.,  10.,   6.,   8.,   7.,  10.,   8.,  10.,   9.,   9.,
         10.,   8.],
       [ 12.,  14.,  12.,  15.,  15.,  11.,  14.,  14.,  11.,  13.,  13.,
         16.,  15.,  11.,  14.,  15.,  14.,  16.,  14.,  13.,  15.,  11.,
         11.,  14.]])
In [5]:

import scipy.interpolate as si
In [6]:

Q=si.NearestNDInterpolator(data1[:,~(np.isnan(data.reshape((3,-1))).any(0))][[1,2]].T, 
                           data1[:,~(np.isnan(data.reshape((3,-1))).any(0))][0])
In [8]:
#the first value is the answer, the 2nd is the index of the nearest neighbor.
Q.tree.query([5,21])
Out[8]:
(6.082762530298219, 3)