Big O Notation - Growth Rate

Question

I am trying to understand if my reasoning is correct:

If I am given the following snippet of code and asked to find it's Big O:

 for(int i = 3; i < 1000; i++)
    sum++;

I want to say O(n) because we are dealing with one for loop and sum++ which is iterated say n times but then looking at this I realise we are not dealing with n at all as we are given the amount of times this for loop iterates... but in my mind it would be wrong to say that this has a Big O of O(1) because the growth is linear and not constant and depends on the size of this loop (although the loop is 'constant'). Would I be correct in saying that this is O(n)?

Also, another one that has me thinking around which has a similar setup:

 for(int i = 0; i < n * n * n; i++)
    for(int j = 0; j < i; j++)
       sum++;

Now here again I know that when dealing with a nested loop containing and outer and inner loop we would use the multiplication rule to derive our Big O. Let's assume that the inner loop was in fact j < n then I would say that the Big O of this snippet of code is O(n^4) but as it isn't and we have a the second loop running its iterations off i and not n then would it be correct to say this as a Big Order of O(n^3)?

I think what is throwing me is where 'n' is not appearing and we're given a constant or another variable and all of a sudden I'm assuming n must not be considered for that section of code. However, having said that the other part of my reasoning is telling me that despite not seeing an 'n' I should still treat the code as though there were an n as the growth rate would be the same regardless of the variable?

Answer 1

It works best if you consider the code to always be within a function, where the function's arguments are used to calculate complexity. Thus:

// this is O(1), since it always takes the same time
void doSomething() {
    for(int i = 3; i < 1000; i++)
        sum++;
}

And

// this is O(n^6), since it only takes one argument
// and if you plot it, the curve matches t = k * n^6
void doSomethingElse(int n) {
  for(int i = 0; i < n * n * n; i++)
     for(int j = 0; j < i; j++)
        sum++;
}

In the end, the whole point of big-O is to say what the run-times (or memory-footprints; but if you don't say anything, you are referring to run-times) look like as the problem size increases . It matters not what happens in the inside (although you can use that to estimate complexity) - what really matters is what you would measure outside.

Looking closer at your second snippet, it's O(n^6) because:

• outer loop runs exactly n^3 times; inner loop runs, on average, n^3 / 2 times.
• therefore, inner sum runs n^3 * k * n^3 times (with ka constant). In big-O notation, that's O(n^6).

Answer 2

The first is either O(1) or simply a wrong question, just like you understand it.

The second is O(n ⁶ ) . Try to imagine the size of the inner loop. On first iteration, it will be 1. On the second, 2. On the i th, it will be i , and on the last, it will be n*n*n . So it will be n*n*n/2 , but that's O(n*n*n) . That, times the outer O(n ³ ) is O(n ⁶ ) overall.

Answer 3

Although the calculation of O() for your question, by others, may be correct, here is a little more insight that should help delineate the conceptual outlook for this whole asymptotic analysis story.

I think what is throwing me is where 'n' is not appearing and we're given a constant or another variable and all of a sudden I'm assuming n must not be considered for that section of code.

The simplest way to understand this one is to identify if the execution of a line of code is affected by/related to the current value of n. Had the inner loop been, let's say, j < 10 instead of j < i, the complexity would have well been O(n^3).

Why is any constant considered O(1)?

This may agreeably sound a little counter-intuitive at first however, here is a small conceptual summary to clear the air. Let us say that your first loop runs 1000 times. Then you set it to 10^1000 times and try to believe that hey, it doesn't take the same time anymore. Fair enough! Even though it may now take your computer 5 seconds more to run the same piece of code, the time complexity still remains O(1). What this practically means is that you can actually calculate the time that it takes your computer to execute that piece of code and it will remain constant forever (for the same configuration).

Big-Oh is actually a function on the input and not the measure of the discrete value itself (time/space).

I hope that the above explanation also helps clarify why we actually ignore the constants in the O() notation.

Why is this Big-Oh thing so generalized and why is it used at the first place?

I thought of including this extra info as I myself had this question in mind when learning this topic for the first time. Asymptotic time-complexity is an apriori analysis of any algorithm to understand the worst (Big-Oh) behavior (time/space) of that program regardless of the size of the input. Eg. Your second code can not perform worse than O(n^6). It is generalized because from one computer to another, only the constant changes, not Big-Oh.

With more experience, you will realize that practically, you would want your algorithm's time-complexity to be as asymptotically small as possible. Till a polynomial function it is fine. But for large inputs, today's computers start coughing if you try to run an algorithm with exponential time complexity of the order O(k^n) or O(n^n), eg. The Travelling Salesman and other NP-C/H problems.

Hope this adds to the info. :)