How do I get all the rows meeting some criteria out of a big matrix?

Question

I have a big matrix (on the order of 1GB) where each row represents the (x,y) coordinate of a point I sampled a surface at, and its z-height.

How do I get all the points less than some euclidean distance away from the (x,y) coordinate?

Currently I am doing this awful thing:

% mtx_pointList is a large matrix with each row containing a sample: [x y z]. 
% We want to get all the samples whose (x,y) point is less than dDistance 
%   away from the vector: v_center = [x y].


mtx_regionSamples = zeros(0,3); for k=1:length(mtx_pointList(:,1)) if( norm( mtx_pointList(k,[1 2])-v_center ) < dDistance^2 ) mtx_regionSamples = [ mtx_regionSamples mtx_pointList(k,:) ] end end 

...but in my application this loop would have to be run around 250k times.

How do I make it do the same thing faster?

Answer 1

Use pdist2 (its default option is Euclidean distance):

ind = pdist2(mtx_pointList(:,[1 2]), v_center) < dDistance; %// logical index
result = mtx_pointList(ind,:);

If the matrix is too large, divide it into chunks of as many rows as your memory allows, and loop over the chunks.

Answer 2

bsxfun

If you don't have pdist2 (statistics toolbox), here's one way to compute distances with bsxfun :

da = bsxfun(@minus,mtx_pointList(:,[1 2]),permute(v_center,[3 2 1]));
distances = sqrt(sum(da.^2,2));

Then find point that meet your criteria:

distThresh = 0.5; % for example
indsClose = distances < distThresh
result = mtx_pointList(indsClose,:);

alternative

You can also use an alternate form of Euclidean (2-norm) distance,

||A-B|| = sqrt ( ||A||^2 + ||B||^2 - 2*A.B )

In MATLAB code:

a = mtx_pointList(:,[1 2]); b = v_center;
aa = dot(a,a,2); bb = dot(b,b,2); ab=a*b.'; %' or sum(a.*a,2)
distances = sqrt(aa + bb - 2*ab); % bsxfun needed if b is more than one point

As Luis Mendo points out, the sqrt is not necessary if you threshold against distThresh^2 .