迭代和递归算法的时间复杂度

Question

I have problem understanding time complexity of algorithms.

Let's for first example take this algorithm for searching in binary search tree:

def search_iteratively(key, node): 
     current_node = node
     while current_node is not None:
         if key == current_node.key:
             return current_node
         elif key < current_node.key:
             current_node = current_node.left
         else:  # key > current_node.key:
             current_node = current_node.right
     return None

So, how to calculate this time complexity?

Let's take for example this recursive algorithm:

int f(int a, int b) 
{ 
    if (a > 0)
        return f(a − 1, b − 3); 
    else 
        return b;
}

So, I presume that time complexity of this algorithm is O(a), because end condition depends only on a parameter.

If I write this down:

T(a, b) = O(1) where a <= 0
T(a, b) = T(a-1, b-3) where a > 0

T(a, b) = 
T(a-1, b-3) = 
T(a-1, b-3) + T(a-2, b-6) = 
T(a-1, b-3) + T(a-2, b-6) + T(a-3, b-9)

So, how do I know that this is linear time complexity? Just because the recursion will end when a is less than 1?

And for the end:

• is it true that we can convert every recursive algorithm into iterative?
• is it the speed of recursive algorithms normally slover than than iterative?
• can we substitute tail recursion with loop?

Answer 1

As for the complexity of the tree search algorithm - try to think of something that changes on each iteration. TIP: think of the depth of current_node in the tree.

Try to use induction to prove linear complexity in this particular case. You know that T(0, x) will end with a single call and this would be your base. Try proving that T(n, x) will perform n recursive calls.

• There is a theorem that every iterative algorithm can be converted to recursion and vise versa
• If you implement the same algorithm without any optimization recursively and iteratively recursion will be slower because there are function calls - a new frame will have to be allocated on the stack and then it will have to be popped
• It is relatively easy to substitute tail recursion with loop in most cases

Answer 2

What is the worst-case time complexity of finding a value in a binary search tree? The worst case is when you have to descend to the deepest leaf. In general, a binary tree of n nodes can have depth O(n) . (Think of a case where every right child is a leaf while the left children descend ever downward.) However, if you maintain a balanced binary search tree such as a red-black tree , you are guaranteed a height of O(log n) . That is the worst-case running time of the key-find operation in a red-black tree.

Your function f is defined as:

• f(a, b) = f(a − 1, b − 3) if a > 0
• f(a, b) = b otherwise

We can prove by induction on a that evaluating f(a, b) for any non-negative value of a requires a calls to f . In the base case, with a == 0 , f is called just once. For positive a , assume that f(a - 1, b) is called a - 1 times. Then evaluating f(a, b) requires a - 1 + 1 = a calls to f . (By the way, we can observe that f(a, b) = b - 3*a and arrive at a constant-time implementation.)

Every recursive algorithm can be converted into an iterative algorithm that simulates a stack on which recursive function calls are executed. Observe that the computer performs iteration to implement your recursive program. More profoundly, Turing machines are iterative. It is an axiom of computer science that everything that can be computed can be computed with a Turing machine. The lambda calculus offers no greater computational power than the Turing machine.

Recursive algorithms generally take more time and space than iterative algorithms because they require allocating a new frame on the stack for each function call.

If a recursive function is written in such a way that every function call is in tail position, meaning that the call does not return an intermediate value requiring further computation, it is a tail-recursive function. The recursive computation does not depend on any values other than the arguments to the recursive call. Thus, the final call to the function immediately produces the final result and there is no need to go back up the chain of recursive calls.

A compiler can implement tail recursion in such a way that the current frame is reused instead of having to allocate a new frame on the stack. Scheme compilers, for example, are required to do this. The resulting computation has the performance characteristics of iteration, yet the code has the expressive advantages of recursion.

Answer 3

The binary tree algorithm visits each node in the tree at most once; otherwise there is a loop in the binary tree, a contradiction. Furthermore in the worst case it visits every node at least once. Therefore the number of visits to nodes is Theta(n) and, since each visit requires Theta(1) time, the worst-case running time is Theta(n).

The way we "know" the solution to a recurrence is by inductive proof. In your case, the basis is

T(0,b) = c for any b

The induction hypthesis is T(n,b) <= c(n+1) for any b.

The induction step is

T(n,b) <= c + T(n-1,b-3) <= c + cn = c(n+1).

It follows that T(n) = O(n).

No, a recursive algorithm is not necessarily slower, ane yes, a loop can be replaced by tail recursion.