Is it possible to reduce a quadratic (or higher) complexity to linear?

Question

I'm working on the processing of a video file to find similarities between 2 frame blocks, assume F and G, which are represented as a 1D array of elements, both of a same size n which can be interpreted as a square of pixels of width W and height H, where W and H are different.

To find the frame block similarities I should be using the sum of absolute differences, SAD, which is the sum of all the differences of pixels at the same position of each frame given a block B of size Bw x Bh. As extra, I also need the average of all SAD values.

Considering that given the values for W, H, Bw and Bh it is possible to find all the blocks in a row and all the blocks in a column, and thus all blocks of a size Bw x Bh by multiplying those two values, the quickest implementaion I thought of was this:

Using a sad_total variable, loop through all blocks in a column, then all blocks in each row, where a variable to get the curr_sad for a single block gets it by looping through all Bh columns and all Bw rows of a block, where curr_sad is obtained via the absolute value of the elements' subtraction for both frames at the specified indices, then added to sad_total then repeated until it has all, then divided by the total number of blocks to get the average.

Assuming Bw and Bh are the same, in terms of Bw, I get a complexity of O(n^4) this way. Considering I cannot sort as I need to subtract each element in order, is there a way to reduce the nesting of the loops?

Please see the attached image for clarity, and thank you to anyone who may provide help

Answer 1

Consider using cumulative sums - calculate matrix of sum of all differences in rectangles starting from top left corner 0,0 and ending in i,j . It might be calculated in O(n^2) relative to matrix size n (linear relative to element count) ( Python example )

Then sum of values in rectangle (x1,y1)-(x2,y2) is equal to sum(x2,y2) + sum(x1,y1)-sum(x2,y1)-sum(x1,y2) , so O(1) for every window, and O(n^2) for the whole matrix

For example, look at OpenCV description of integral function (perhaps you even can use OpenCV library for your purposes).

Answer 2

First calculate matrix sum for all i, j varying from 0, 0 to W-1, H-1 . This can be done in O(W*H) using dynamic programming. The recurrence relation for which is like F(i, j) = F(i, j-1) + F(i-1, j) - F(i-1, j-1) + M[i][j] . Then we can create a 2D Fenwick Tree and insert these values in it. Then for i, j in this tree, we can get the cumulative sum in O(log(W)*log(H)) . Hence for all i, j , total time would be O(W*H*log(W)*log(H)) .

My code for these two elementary steps:

#include <iostream>
#include <vector>

using namespace std;

int sum(const vector<vector<int>>& BIT2D, int x, int y)
{
    int res = 0;

    for (int i = x; i > 0; i -= i & -i)
        for (int j = y; j > 0; j -= j & -j)
            res += BIT2D[i][j];
    return res;
}

void add(vector<vector<int>>& BIT2D, int x, int y, int value)
{
    for (int i = x; i < BIT2D.size(); i += i & -i)
        for (int j = y; j < BIT2D[i].size(); j += j & -j)
            BIT2D[i][j] += value;
}

int foobar(const vector<vector<int>>& matrix, vector<vector<int>>& dp, int i, int j)
{
    if (dp[i][j] != -1)
        return dp[i][j];

    if (i == 0 || j == 0)
        return dp[i][j] = 0;
    return dp[i][j] = foobar(matrix, dp, i, j - 1) + foobar(matrix, dp, i - 1, j) - foobar(matrix, dp, i - 1, j - 1) + matrix[i - 1][j - 1];
}

int main()
{
    int N, M;
    cin >> N >> M;
    vector<vector<int>> matrix(N, vector<int>(M)), dp(N + 1, vector<int>(M + 1, -1));
    for (int i = 0; i < matrix.size(); ++i)
        for (int j = 0; j < matrix[i].size(); ++j)
            cin >> matrix[i][j];
    foobar(matrix, dp, N, M);

    vector<vector<int>> BIT2D(N + 1, vector<int>(M + 1));
    for (int i = 1; i < dp.size(); ++i)
        for (int j = 1; j < dp[i].size(); ++j)
            add(BIT2D, i, j, dp[i][j]);
    for (int i = 1; i < BIT2D.size(); ++i)
    {
        for (int j = 1; j < BIT2D[i].size(); ++j)
            cout << sum(BIT2D, i, j) << ' ';
        cout << endl;
    }
}

Do the same for the 2nd matrix and then we can calculate SAD . Infact, you can do the same computation for the 2nd matrix (as well as calculate SAD ) within one call of foobar if you modify it right. The time complexity will stay O(W*H*log(W)*log(H)) .

Edit: The cumulative sums can be calculated in O(W*H) . No need to create the Fenwick Tree. Call foobar again on the dp passed to the first call of foobar and store the results in dp2 .

Answer 3

It seems I misunderstood my own implementation of this operation. Even if we assume Bh and Bw are the same, the complexity is not actually in a degree of 4. From the pseudocode:

  // assuming the 1D array width is w, the height is h, and both Bh and Bw is b:

  for (int i = 0; i < h; i += b) {
    for (int j = 0; j < w; j += b) {
      for (int y = 0; y < b; y++) {
        for (int x = 0; x < b; x++) {
          // code
        }
      }
    }
  }

In terms of b , we can tell the two innermost loops are both O(b) . But the two outermost loops are different. Given the step/update value of b , the complexity for the outermost is O(h / b) , and O(w / b) for the other one. Thus, the actual complexity would be:

O((h / b) * (w / b) * b * b ) = O(h * w)

And, since h * w is the actual size of the 1D array of elements used for this, it makes sense the implementation has this linear complexity in terms of the array size, because to get the average of all blocks I need to access all indices.