如何降低我的O​​（N ^ 2）C指数函数的时间复杂度？

Question

I have the following function (in Matlab) which will calculate the concordance index for a given set of predictions and observed values:

function civalue = CI(predval)
% FUNCTION civalue = CI(predval)
%
% DESCRIPTION: 
% - This function will calculate the concordance index. Not suitable for
% big vectors. O(n^2) time function. 
%
% INPUTS: 
% 'predval' a n-by-2 matrix, where the first column consists of the
% prediction values and the second column the actual label values. 
%
% OUTPUT: 
% 'civalue' the CI-value.

N = 0;
hSum = 0;

for i = 1:size(predval, 1)

    yi_pred = predval(i, 1);
    yi_val = predval(i, 2);
    for j = i+1:size(predval, 1)
        yj_pred = predval(j, 1);
        yj_val = predval(j, 2);
        if yi_val ~= yj_val
            N = N + 1;
            if  (yi_pred < yj_pred && yi_val < yj_val) || (yi_pred > yj_pred && yi_val > yj_val) % Order correct
                hSum = hSum + 1;
            elseif (yi_pred < yj_pred && yi_val > yj_val) || (yi_pred > yj_pred && yi_val < yj_val) % Order opposite 
                hSum = hSum + 0;
            elseif yi_pred == yj_pred % Random
                hSum = hSum + 0.5;
            end
        end
    end

end

civalue = hSum / N;

My function has a time complexity of O(N^2). The idea of the code is to do pairwise comparisons between data points. Any ideas how I could reduce the time complexity of my code?

The idea behind the CI-value or C-index, is to measure how well a prediction model was able to rank data points into correct order. What you give to this function is a set of observed values X and their corresponding predictions Y. The function will do the ranking comparison between data points that have different observed values, because they obviously have a ranking.

For example, lets say you have a two observed values for some variable, eg stock price: P1 = 5$, P2 = 7$

Now we create a model that will try to predict the stock prices. Lets say we builded our model and tested its ability to predict the stock price and for the two data points P1, P2 it predicted the values Y1 = 5.5$ and Y2 = 8$.

Now you can see that the model got the ORDER correct, P1 < P2 && Y1 < Y2 but not the absolute value. This is useful when we need to make selections between a set of alternatives, eg which stock should I buy that will increase in value most etc.

Thank you for all help! Please let me know if you need any more information etc. :)

Here is the comparison between my own and Martin's implementation:

Answer 1

You can significantly improve the run-time by vectorizing the inner loop. The code below can be optimized further (at the expense of legibility). On my machine, using random input, the code runs about 50x faster and produces the same results. (Random input is probably a bad test-case, as the == branches will never execute)

N = 0;
hSum = 0;
for i = 1:size(predval, 1)

    yi_pred = predval(i, 1);
    yi_val = predval(i, 2);
    yj_pred = predval(i+1:end,1);
    yj_val = predval(i+1:end,2);
    idxs = yi_val ~= yj_val;
    N = N + sum(idxs);

    yj_pred = yj_pred(idxs); % redefined to make the next lines prettier
    yj_val = yj_val(idxs); 
    hSum = hSum + sum((yi_pred < yj_pred & yi_val < yj_val) | ...
        (yi_pred > yj_pred & yi_val > yj_val)); % Order correct
    hSum = hSum + 0.5*sum(yi_pred == yj_pred); % Order random
end

The complexity of the function is still O(n^2), though.

Answer 2

Assuming that your final goal is to improve the runtime performance and if you have good enough memory to run a vectorized approach, this could be one of those -

%// Column arrays
c1 = predval(:,1);
c2 = predval(:,2);

%// Get logical arrays of IF conditional statements in the original code
start_cond = bsxfun(@ne,c2,c2.')               %//'# starting condition

%// Rest of the three IF conditionals
case1 = bsxfun(@lt,c1,c1.') & bsxfun(@lt,c2,c2.') | ...
    bsxfun(@gt,c1,c1.') & bsxfun(@gt,c2,c2.')  %//'
case2 = bsxfun(@lt,c1,c1.') & bsxfun(@gt,c2,c2.') | ...
    bsxfun(@gt,c1,c1.') & bsxfun(@lt,c2,c2.')  %//'
case3 = bsxfun(@eq,c1,c1.')                    %//'

%// Get the counts for different cases and finally get the output sum
w1 = start_cond & case1
w2 = start_cond & ~case1 & ~case2 & case3
hSum = sum(w1(:))./2 + sum(w2(:))./4