Matlab from text file to sparse matrix.

Question

I have a huge text file in the following format:

1   2
1   3
1   10
1   11
1   20
1   376
1   665255
2   4
2   126
2   134
2   242
2   247

First column is the x coordinate while second column is the y coordinate. It indicates that if I had to construct a Matrix

M = zeros(N, N);
M(1, 2) = 1;
M(1, 3) = 1;
.
.
M(2, 247) = 1;

This text file is huge and can't be brought to main memory at once. I must read it line by line. And save it in a sparse matrix .

So I need the following function:

function mat = generate( path )
    fid = fopen(path);
    tline = fgetl(fid);
    % initialize an empty sparse matrix. (I know I assigned Mat(1, 1) = 1)
    mat = sparse(1);
    while ischar(tline)
        tline = fgetl(fid);
        if ischar(tline)
            C = strsplit(tline);
        end
        mat(C{1}, C{2}) = 1;
    end
    fclose(fid);
end

But unfortunately besides the first row it just puts trash in my sparse mat. Demo:

1 7
1 9
2 4
2 9

If I print the sparse mat I get:

   (1,1)        1
  (50,52)       1
  (49,57)       1
  (50,57)       1

Any suggestions ?

Answer 1

Fixing what you have...

Your problem is that C is a cell array of characters , not numbers. You need to convert the strings you read from the file into integer values. Instead of strsplit you can use functions like str2num and str2double . Since tline is a space-delimited character array of integers in this case, str2num is the easiest to use to compute C :

C = str2num(tline);

Then you just index C like an array instead of a cell array:

mat(C(1), C(2)) = 1;

Extra tidbit: If you were wondering how your demo code still worked even though C contained characters, it's because MATLAB has a tendency to automatically convert variables to the correct type for certain operations. In this case, the characters were converted to their double ASCII code equivalents: '1' became 49 , '2' became 50 , etc. Then it used these as indices into mat .

A simpler alternative...

You don't even have to bother with all that mess above, since you can replace your entire function with a much simpler approach using dlmread and sparse like so:

data = dlmread(filePath);
mat = sparse(data(:, 1), data(:, 2), 1);
clear data;  % Save yourself some memory if you don't need it any more