具有大型稀疏矩阵的MemoryError

Question

For a project I have built a program that constructs large matrices.

def ExpandSparse(LNew):
SpId = ssp.csr_matrix(np.identity(MS))
Sz = MS**LNew
HNew = ssp.csr_matrix((Sz,Sz))
Bulk = dict()
for i in range(LNew-1):
    for j in range(LNew-1):
        if i == j:
            Bulk[(i,j)]=H2
        else:
            Bulk[(i,j)]=SpId
Ha = ssp.csr_matrix((8,8))
try:
    for i in range(LNew-1):
        for j in range(LNew-2):
            if j < 1:
                Ha = ssp.csr_matrix(ssp.kron(Bulk[(i,j)],Bulk[(i,j+1)]))
            else:
                Ha = ssp.csr_matrix(ssp.kron(Ha,Bulk[(i,j+1)]))
        HNew = HNew + Ha
except MemoryError:
    print('The matrix you tried to build requires too much memory space.')
    return
return HNew

This does the job, however it does not work as well as I would have expected. The problem is that it won't allow for really large matrices. When LNew is larger than 13 I will get a MemoryError. My experiences with numpy suggest that, memorywise, I should be able to get LNew up to 18 or 19 before I get this error. Does this have to do with my code, or with the way scipy.sparse.kron() works with these matrices?

Another note that might be important is that I use Windows not Linux.

Answer 1

After some more reading on the working of the scipy.sparse.kron() function I have noticed that there is a third term named format you can enter. The default setting is None, but when it is put on 'csr' or another supported format it will only use the sparse format making it a lot more efficient, now for me it can build a 2097152 x 2097152 matrix. Here LNew is 21.