On a computer with 4GB of memory this simple interpolation leads to a memory error:
(based on: http://docs.scipy.org/doc/scipy/reference/tutorial/interpolate.html )
import numpy as np
from scipy.interpolate import interp1d
x = np.linspace(0, 10, 80000)
y = np.cos(-x**2/8.0)
f2 = interp1d(x, y, kind='cubic')
I thought about cutting the data into chunks, but is there a way I can perform this cubic spline interpolation without requiring so much memory? Why does it even get in trouble?
If you look at the traceback when the error occurs, you'll see something like:
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-4-1e538e8d766e> in <module>()
----> 1 f2 = interp1d(x, y, kind='cubic')
/home/warren/local_scipy/lib/python2.7/site-packages/scipy/interpolate/interpolate.py in __init__(self, x, y, kind, axis, copy, bounds_error, fill_value)
390 else:
391 minval = order + 1
--> 392 self._spline = splmake(x, y, order=order)
393 self._call = self.__class__._call_spline
394
/home/warren/local_scipy/lib/python2.7/site-packages/scipy/interpolate/interpolate.py in splmake(xk, yk, order, kind, conds)
1754
1755 # the constraint matrix
-> 1756 B = _fitpack._bsplmat(order, xk)
1757 coefs = func(xk, yk, order, conds, B)
1758 return xk, coefs, order
MemoryError:
The function that is failing is scipy.interpolate._fitpack._bsplmat(order, xk)
. This function creates a 2-d array of 64-bit floats with shape (len(xk), len(xk) + order - 1)
. In your case, this is over 51GB.
Instead of interp1d
, see if InterpolatedUnivariateSpline
works for you. For example,
import numpy as np
from scipy.interpolate import InterpolatedUnivariateSpline
x = np.linspace(0, 10, 80000)
y = np.cos(-x**2/8.0)
f2 = InterpolatedUnivariateSpline(x, y, k=3)
I don't get a memory error with this.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.