Sparse Matrix Addition yields 'ValueError: setting an array element with a sequence.'

Question

The lines in question are:

# Make efficient matrix that can be built
K = sparse.lil_matrix((N, N))

# Calculate K matrix (<i|pHp|j> in the LGL-nodes basis)
for i in range(Ne):
    idx_s, idx_e = i*(Np-1), i*(Np-1)+Np
    print(shape(K[idx_s:idx_e, idx_s:idx_e]))
    print(shape(dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np)).dot(dmat)))
    K[idx_s:idx_e, idx_s:idx_e] += dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np)).dot(dmat)

But, currently, Numpy is yielding the error

(8, 8)
(8, 8)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-62-cc7cc21f07e5> in <module>()
     22 
     23 for _ in range(N):
---> 24     ll, q = getLL(Ne, Np, x_d, w_d, dmat_d, x, w, dL, peq*peq, data)
     25     peq = (peq*q)
     26 

<ipython-input-61-a52c13d48b87> in getLL(Ne, Np, x_d, w_d, dmat_d, x, w, dmat, peq, data)
     15         print(shape(K[idx_s:idx_e, idx_s:idx_e]))
     16         print(shape(dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np)).dot(dmat)))
---> 17         K[idx_s:idx_e, idx_s:idx_e] += dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np)).dot(dmat)
     18 
     19     # Re-make matrix for efficient vector products

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scipy/sparse/lil.py in __iadd__(self, other)
    157 
    158     def __iadd__(self,other):
--> 159         self[:,:] = self + other
    160         return self
    161 

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/scipy/sparse/lil.py in __setitem__(self, index, x)
    307 
    308         # Make x and i into the same shape
--> 309         x = np.asarray(x, dtype=self.dtype)
    310         x, _ = np.broadcast_arrays(x, i)
    311 

/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/numpy/core/numeric.py in asarray(a, dtype, order)
    460 
    461     """
--> 462     return array(a, dtype, copy=False, order=order)
    463 
    464 def asanyarray(a, dtype=None, order=None):

ValueError: setting an array element with a sequence.

This is a little cryptic as it seems that the error is happening somewhere inside of the Numpy library---not in my code. But I'm not terribly familiar with numpy, per se, so perhaps I'm indirectly causing the error.

Both slices are of the same shape, so that doesn't seem to be the actual error.

Answer 1

The problem is that

(dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np)).dot(dmat)

is not a simple array. It has the right shape, but the elements are sparse matrices (the 'sequence' in the error message).

Turning the inner sparse matrix into a dense array should solve the problem:

dmat.T.dot(sparse.spdiags(w*peq[idx_s:idx_e], 0, Np, Np).A).dot(dmat)

The np.dot method is not aware of sparse matrices, at least not in your version of numpy (1.8?), so it treats it as sequence. Newer versions are 'sparse' aware.

Another solution is to use the sparse matrix product ( dot or * ).

sparse.spdiags(...).dot(dmat etc)

I had to play around to get reasonable values for N,Np,Ns, dmat,peq . You really should have given us small samples. It makes testing ideas much easier.