Surface Plot in Python - MemoryError

Question

I'm trying to create a surface plot using coordinates using the following code:

from mpl_toolkits.mplot3d import Axes3D
from scipy.stats import gaussian_kde

fig = plt.figure()
ax = Axes3D(fig)
X = []
Y = []
XY = np.vstack([X, Y])
x, y = np.meshgrid(X, Y)
z = gaussian_kde(XY)(XY).tolist()
ax.plot_surface(x, y, z)
plt.show()

Assume that X and Y are long lists of floating point numbers. However, when I execute the code I get a MemoryError, which traces back to the meshgrid method: in meshgrid return [x * mult_fact for x in output] .

Does anyone know how to solve this?

(When I alter the code to use a subset of the lists by writing XY = np.vstack([X[0:10], Y[0:10]]) and x, y = np.meshgrid(X[0:10], Y[0:10]) the surface plot works, but overlaps. I suppose this is due to the points not being plotted in order. Does anyone know how to resolve this also? I am new to Python...)

Answer 1

You are trying to plot too many points. To see this give your example some fake data:

from mpl_toolkits.mplot3d import Axes3D
from scipy.stats import gaussian_kde
import numpy as np
import matplotlib.pyplot as plt

fig = plt.figure()
ax = Axes3D(fig)

npoints = 100
X = np.linspace(0, 1, npoints)
Y = np.sin(X)
XY = np.vstack([X, Y])
x, y = np.meshgrid(X, Y)
z = gaussian_kde(XY)(XY).tolist()
ax.plot_surface(x, y, z)
plt.show()

This will work just fine, but if we make npoints=10000 then (on my machine) I can reproduce your error.

The solution depends on exactly what X and Y are. For example if they are some measured data output then you need to ask the question "do I need to plot all of this data?". In my experience if your trying to plot so many points that you get a memory error, then you won't be able to see all of the data when it is plotted anyway! So, you could down sample it, just pick every n th point. First I would suggest turning X and Y into numpy arrays

import numpy as np
X = np.array(X)
Y = np.array(Y)

Then, to get every 100th point do:

X = X[::100]
Y = Y[::100]

For more information on array slicing and indexing see the docs .

Alternatively you may feel that this "looses" some valuable information, in which case you could apply a rolling window average then separately consider the variations in each window to check you haven't missed anything. Ultimately it will all depend on what the data is.