包含并行性的 Big-O 符号版本？

Question

This question requires some set up.

http://igoro.com/archive/big-oh-in-the-parallel-world/

"Method 1" in this link details a notation for describing the effect of parallelism on algorithmic time complexity. I really like this method of description, but this is the only place I have ever found it.

My question is as follows: why is this notation not used more often?

One explanation is actually given in the method description itself: "To multiply two 100×100 matrices, we would need a machine with a 10,000 processors". This quote seems to suggest that 10,000 processors is totally unreasonable for anyone to expect.

My rebuttal:

https://en.wikipedia.org/wiki/Manycore_processor

The largest super computer in the world right now, Taihu Light, contains a total of 40960*256 = 10485760 cores. We don't all have access to a super computer like Taihu Light, but with cloud computing continuously gaining popularity, I see no reason why 10,000 core processors and beyond won't be widely available for use in the near future. On top of that, the average GPU has 1000s of cores already. They're more specialized than CPU cores, but that's besides the point.

Since there are so few examples of this notation, I'll also provide my own example using an equivalent description, which I call big money notation.

Say we have a perfectly balanced tree with a branching factor of b. All nodes save one contain a boolean value of false, and the last contains a boolean value of true. We want to find and return this true node.

Using a breadth first search, the the Big-O runtime to find the true node would be O(|V|+|E|) == O(b^d), where b is the branching factor and d is the depth of the tree.

However, if we assume that we have infinitely many processing units, each with finite processing capabilities, the big-money notation is as follows: $O(b^d -> d). What this says is that on 1 processor, the algorithm would take O(b^d) time, but as the number of processors increases, the time complexity approaches d, the depth of the tree. This is because at each false node, we can spawn b more processes to search each of that nodes b children. Since we have infinite cores, all of these processes can work in parallel. This is why I call it big money notation. Time complexity decreases with more processing power, processing power increases as you throw more money at Amazon/Microsoft/Google/YourFavoriteCloudProvider.

I'll reiterate my question again. Why is notation similar to this not seeing more widespread use? Thanks!

Answer 1

The complexity class NC deals with problems that can be solved efficiently in a parallel setting. As far as I know, people who work on parallel algorithms do use the traditional O(.) notation and use expressions like "NC algorithms", so when you read their papers and see sentences like:

"We give a O(xxx) NC algorithm for SOMEPROBLEM."

you should interpret this as:

"We give a parallel algorithm for SOMEPROBLEM which solves it in time O(xxx) using O(yyy) processors."

(where xxx and yyy vary of course).

I don't recall seeing something that resembles your notation, but I'm not sure what it would bring to the table. Complexity classes are often partitioned according to some parameter and / or some measure of growth of that parameter (eg, we have access to O(1) processors, or O(log n) processors, or ...) so as to classify problems with a degree of precision that your notation seems to lack if I interpret it correctly.