Python: Iterate through list and remove duplicates (without using Set())

Question

So I have a list:

s = ['cat','dog','cat','mouse','dog']

And I want to be able to iterate through the list and remove the duplicates, WITHOUT using the set() function! So for example it should remove 'cat' and position s[2] BUT keep 'cat' at position s[0]. It then needs to do the same thing for 'dog', ie. keep 'dog' at position s[1] but remove 'dog' from position s[4].

So the output is then:

s = ['cat','dog','mouse']

I have tried to use i and j as index positions in the list, and check whether the element at position i is equal to the element at position j. If so, it will remove it and increment the value of j by 1, if not then it will leave it and just increment the value of j. After the whole list has been iterated through, it will increment the value of i and then check the whole list again, for the new element. Below:

i = 0
j = 1
for a in range(len(s)):
    for b in range(len(s)):
        if s[i] == s[j]:
            s.remove(s[j])
            j = j + 1
        else:
            j = j + 1
    i = i + 1

What am I doing wrong here?

Answer 1

The issue is with "automatic" for loops - you have to be careful about using them when modifying that which you are iterating through. Here's the proper solution:

def remove_dup(a):
   i = 0
   while i < len(a):
      j = i + 1
      while j < len(a):
         if a[i] == a[j]:
            del a[j]
         else:
            j += 1
      i += 1

s = ['cat','dog','cat','mouse','dog']
remove_dup(s)
print(s)

Output: ['cat', 'dog', 'mouse']

This solution is in-place, modifying the original array rather than creating a new one. It also doesn't use any extra data structures.

Answer 2

You can loop through the list and check if the animal has already been added.

s = ['cat','dog','mouse','cat','horse','bird','dog','mouse']

sNew = []
for animal in s:
    if animal not in sNew:
        sNew.append(animal)

s = sNew

Answer 3

You shouldn't alter the list while you iterate over it, you'll likely either skip elements or get an IndexError . If you just can't use set use collections.OrderedDict :

>>> from collections import OrderedDict

>>> s = ['cat','dog','cat','mouse','dog']

>>> list(OrderedDict.fromkeys(s).keys())
['cat', 'dog', 'mouse']

Answer 4

Here's a one line solution:

s = ['dog', 'cat', 'cat', 'mouse', 'dog']   

answer = [animal for idx, animal in enumerate(s) if a not in s[:idx]]

And you'll see:

>>> answer
['cat', 'dog', 'mouse']

Answer 5

I am not sure why you wouldn't use a set, but here is an alternative. Iterate over your original list, placing each element into a new list if it is not already in the new list. Example:

l = []
s = ['dog', 'cat', 'cat', 'mouse', 'dog']

for i in range(len(s)):
    if s[i] not in l:
        l.append(s[i])

Now:

>>> s
['dog', 'cat', 'mouse']

Answer 6

s = ['cat','dog','cat','mouse','dog']
duplicates = []

for animal in s:
  if s.count(animal) > 1:
    if animal not in duplicates:
      duplicates.append(animal)
print(duplicates)

Answer 7

Here is only by Type-Casting,

s = ['cat','dog','cat','mouse','dog']

l = list(set(s)) 

print(l)