saving numpy matrix in text file with indices

Question

I am trying to save a 2D matrix A to text file for each non-zero entry. I would like to save it in the below format,

For the (i,j)-th entry, one line as

row[i]::column[j]::A[i,j]

where row and column are 1D numpy array corresponding to rows and column indices

I tried,

np.savetxt(rel_file,(row,column,A),fmt='%d',delimiter="::",newline="\n");

But, since shape mismatches, i get error. I do not want to iterate over each row and column indices in a loop (I think it is too time consuming, i have 5000*5000 matrix).

Answer 1

In my first answer I display all values; with just the nonzeros it's even easier to create the column array. np.where does the trick

In [1548]: I,J = np.nonzero(A)    # np.where 
In [1549]: Acol = np.column_stack((I,J,A[I,J]))

==========

For all values:

Here's a start; it is iterative, but as I commented, so is np.savetxt .

In [1523]: A=np.arange(25).reshape(5,5)

In [1526]: list(np.ndenumerate(A))
Out[1526]: 
[((0, 0), 0),
 ((0, 1), 1),
 ((0, 2), 2),
 ((0, 3), 3),
 ((0, 4), 4),
 ((1, 0), 5),

In [1528]: with open('txt','w') as f:
      ...:     for (i,j),v in np.ndenumerate(A):
      ...:         f.write('%d::%d::%d\n'%(i,j,v))
      ...:         
In [45]: cat txt
0::0::0
0::1::1
0::2::2
0::3::3
0::4::4
1::0::5

We could make a (n,3) array consisting of the indices and flattened values of A . But savetxt would still iterate on it, effectively:

with open(...) as f:
   for row in Acol:
      f.write(fmt % tuple(row))

Here's one way of making this 3 column array:

In [1535]: I,J = np.meshgrid(np.arange(A.shape[0]),np.arange(A.shape[1]),indexin
      ...: g='ij')
In [1536]: I
Out[1536]: 
array([[0, 0, 0, 0, 0],
       [1, 1, 1, 1, 1],
       [2, 2, 2, 2, 2],
       [3, 3, 3, 3, 3],
       [4, 4, 4, 4, 4]])
In [1537]: Acol=np.column_stack((I.ravel(),J.ravel(),A.ravel()))
In [1538]: Acol
Out[1538]: 
array([[ 0,  0,  0],
       [ 0,  1,  1],
       [ 0,  2,  2],
       [ 0,  3,  3],
       [ 0,  4,  4],
       ....

Answer 2

Heres a simple method that manually reshapes the original array to be a very long array consisting of row, col, value . Just filter the rows for non-zero and it should work. The below example works with this random 50,50 array.

a = np.random.randint(0,5, (50, 50))

rows = np.repeat(np.arange(50), 50)
cols = np.tile(np.arange(50), 50)

data = np.column_stack((rows, cols, a.flatten()))

data_filtered = data[data[:, 2] != 0]
np.savetxt('temp.txt',data_filtered,fmt='%d',delimiter="::",newline="\n");