Quicksort: Non-in-place implementation works. In-place implementation exceeds maximum recursion depth. Why?

Question

def quicksort_stable(l):
    if not l:
        return l
    else:
        pivot = l[0]
        return quicksort_stable([x for x in l if x < pivot]) \
                + [x for x in l if x == pivot] \
                + quicksort_stable([x for x in l if x > pivot])    

def quicksort_inplace(l):
    def partition(start_idx, end_idx):
        left_idx = start_idx + 1
        right_idx = end_idx
        while True:
            while left_idx <= right_idx and l[left_idx] <= l[start_idx]:
                left_idx += 1
            while right_idx >= left_idx and l[right_idx] >= l[start_idx]:
                right_idx -= 1

            if right_idx < left_idx:
                break
            else:
                l[left_idx], l[right_idx] = l[right_idx], l[left_idx]

        l[start_idx], l[right_idx] = l[right_idx], l[start_idx]     

        return right_idx

    def qs(start_idx, end_idx):
        if start_idx < end_idx:
            split_idx = partition(start_idx, end_idx)
            qs(start_idx, split_idx - 1)
            qs(split_idx + 1, end_idx)

    qs(0, len(l) - 1)
    return l

if __name__ == '__main__':

    import random

    l1 = [random.randint(0, 9) for x in range(10000)]
    l2 = [x for x in l1]

    l1 = quicksort_stable(l1)
    quicksort_inplace(l2)

I deliberately chose the first element as the pivot instead of randomizing to ensure that both implementations behave the same way.

Both implementations are implemented recursively. In the call stack, it seems like quicksort_inplace should take O(lg n) space while quicksort_stable should take O(n) space since it creates a new list every time it recurses.

Yet, quicksort_inplace is the one that causes "maximum recursion depth exceeded" while quicksort_stable works fine.

Why is this so?

Answer 1

I believe the reason for this behavior is that your list contains lots of repetitions (every element appears ~1000 times), and you "cheated" implementing the stable version by gathering all elements equal to pivot at once and not getting back to them (which is of course great!).

So to actually compare these two procedures it should look like this:

    def quicksort_stable(l):
        if not l or len(l)==1:
            return l
        else:
            pivot = l[0]
            rst = l[1:]
            return quicksort_stable([x for x in rst if x < pivot]) \
                    + [pivot] \
                    + quicksort_stable([x for x in rst if x >= pivot])    

Plus, in order to have the destructive (inplace) version of the above you should change a condition on your second while to greater-than (so that on right of right_idx there are elements no less than pivot ), ie

        while right_idx >= left_idx and l[right_idx] > l[start_idx]:

If you do this you'll find that both procedures cause stack overflow on arrays with 10000 elements from range(0,9) (note also that with range(0,99) this is not the case because they require less "cuts").