简体   繁体   中英

Memory Error in Python despite having sufficient RAM

from __future__ import division
import scipy.io 
import numpy as np
import math
from math import sin
from math import cos


mat = np.zeros((1024,1024,360 ),dtype=np.float32)                                    

x = scipy.io.loadmat('/home/prakriti/Project/A.mat')          
A = np.array((x.values()))
mat[:,:,0:120] = A[0][:,:,:]
del x
del A

y = scipy.io.loadmat('/home/prakriti/Project/B.mat')  
B = np.array((y.values()))
mat[:,:,120:240] = B[0][:,:,:]
del y
del B

z = scipy.io.loadmat('/home/prakriti/Project/C.mat')  
C = np.array((z.values()))
mat[:,:,240:360] = C[0][:,:,:]
del z
del C

s = np.linspace(0,1023,1024)
v = np.linspace(0,1023,1024)
ss,vv = np.meshgrid(s,v)
zz = ss**2 + vv**2
print zz

I have been getting the following error for this. Can anyone explain me what is the problem here? I am trying to make a 3-D matrix mat with data that is available to me. I have 49GB of RAM available. Why do I still get memory error?

Traceback (most recent call last):
  File "/home/prakriti/Project/fdk_new.py", line 11, in <module>
    x = scipy.io.loadmat('/home/prakriti/Project/A.mat')
  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio.py", line 152, in loadmat
    matfile_dict = MR.get_variables(variable_names)
  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio5.py", line 270, in get_variables
    hdr, next_position = self.read_var_header()
  File "/usr/lib/python2.7/dist-packages/scipy/io/matlab/mio5.py", line 223, in read_var_header
    stream = BytesIO(dcor.decompress(data))
MemoryError

Providing an answer because a comment would not format correctly.

Try not creating extra memory. The GIL takes a bit of time to release it.

Instead of :

x = scipy.io.loadmat('/home/prakriti/Project/A.mat')          
A = np.array((x.values()))
mat[:,:,0:120] = A[0][:,:,:]
del x
del A

do:

mat[:,:,0:120] = np.array((scipy.io.loadmat('/home/prakriti/Project/A.mat').values()))[0][:,:,:]

etc...

Otherwise, we need more info. Like the size of the mat files.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM