IndexError: List index out of range. It should be in range

Question

The following function removes duplicate elements in a list. I sort the list, then iterate through. If any value is equal to the value that comes after it, delete that value.

def remove_duplicates(myList):
    myList.sort()
    for i in range(len(myList)-2):
        if myList[i] == myList[i+1]:
            del myList[i]
    print(myList)

remove_duplicates([1,1,2,3,4,4,5,6])

Since i will eventually reach the last element in the list and there is no element beyond the last element (ie [i+1] does not exist) then I get an index error.

This error should be remedied by iterating one less time than what would normally be a full iteration, to account for the fact that we are looking ahead of i by 1.

To get the correct output, I believed that I should write

range(len(myList)-2)

because -1 is the last value, and -2 is the value before that.

In fact, this function only works as intended if I write

range(leng(myList)-3)

Why is this the case? It seems that -3 should be one too many? Surely -2 should be enough to account for the [i+1]?

Answer 1

It's failing because you're deleting elements from the list at the same time that you're iterating over it, so the list's length changes, messing with the indexes - there will be less valid indexes. By the way, a more idiomatic solution for removing duplicates (as long as you don't care about the element's order) would be:

 return list(set(myList))

The above uses the fact that a set() eliminates duplicates by definition , so if we build a set from the input list and then a list from the set, we'll have a duplicate-free list - although it's possible that the elements are in a different order, as sets don't guarantee that their elements are kept in the same insertion order.

Answer 2

You don't need to write such deduplicating code from scratch. Just use the built-in set() :

def remove_duplicates(myList):
    return list(set(myList))