python内存密集脚本

Question

After about 4 weeks of learning, experimenting, etc. I finally have a script which does what I want. It changes the perspective of images according to a certain projection matrix I have created. When I run the script for one image it works fine, however I would like to plot six images in one figure. When I try to do this I get a memory error.

All the images are 2448px in width and 2048 px in height each. My script:

files = {'cam1': 'c1.jpg',
         'cam2': 'c2.jpg',
         'cam3': 'c3.jpg',
         'cam4': 'c4.jpg',
         'cam5': 'c5.jpg',
         'cam6': 'c6.jpg'}

fig, ax = plt.subplots()

for camname in files:
    img = Image.open(files[camname])
    gray_img = np.asarray(img.convert("L"))
    img = np.asarray(img)
    height, width, channels = img.shape

    usedP = np.array(P[camname][:,[0,1,3]])
    usedPinv = np.linalg.inv(usedP)
    U, V = np.meshgrid(range(gray_img.shape[1]),
                       range(gray_img.shape[0]))
    UV = np.vstack((U.flatten(),
                    V.flatten())).T
    ones = np.ones((UV.shape[0],1))
    UV = np.hstack((UV, ones))

    # create UV_warped
    UV_warped = usedPinv.dot(UV.T).T

    # normalize vector by dividing by the third column (which should be 1)
    normalize_vector = UV_warped[:,2].T
    UV_warped = UV_warped/normalize_vector[:,None]

    # masks
    # pixels that are above the horizon and where the V-projection is therefor positive (X in argus): make 0, 0, 1
    # pixels that are to far: make 0,0,1
    masks = [UV_warped[:,0]<=0, UV_warped[:,0]>2000, UV_warped[:,1]>5000, UV_warped[:,1]<-5000] # above horizon: => [0,0,1]
    total_mask = masks[0] | masks[1] | masks[2] | masks[3]
    UV_warped[total_mask] = np.array([[0.0, 0.0, 1.0]])

    # show plot
    X_warped = UV_warped[:,0].reshape((height, width))
    Y_warped = UV_warped[:,1].reshape((height, width))
    gray_img = gray_img[:-1, :-1]

    # add colors
    rgb = img[:,:-1,:].reshape((-1,3)) / 255.0 # we have 1 less faces than grid cells
    rgba = np.concatenate((rgb, np.ones((rgb.shape[0],1))), axis=1)

    plotimg = ax.pcolormesh(X_warped, Y_warped, img.mean(-1)[:,:], cmap='Greys')
    plotimg.set_array(None)
    plotimg.set_edgecolor('none')
    plotimg.set_facecolor(rgba)

ax.set_aspect('equal')
plt.show()

I have the feeling that numpy.meshgrid is quite memory intensive, but I'm not sure. Does anybody see where my memory gets eaten away rapidly? (BTW, I have a laptop with 12Gb of RAM, which is only used by other programs for a very small part)

Answer 1

You might want to profile your code with this library .

It will show you where your script is using memory.

Answer 2

There is a Stackoverflow question here about memory profilers. Also, I've used the trick in this answer in the past as a quick way to get an idea where in the code memory is going out of control. I just print the resource.getrusage() results all over the place. It's not clean, and it doesn't always work, but it's part of the standard library and it's easy to do.

Answer 3

I ordinarily profile with the profile and cProfile modules, as it makes testing individual sections of code fairly easy.

Python Profilers