Python list comprehension: list sub-items without duplicates

Question

I am trying to print all the letters in all the words in a list, without duplicates.

wordlist = ['cat','dog','rabbit']
letterlist = []
[[letterlist.append(x) for x in y] for y in wordlist]

The code above generates ['c', 'a', 't', 'd', 'o', 'g', 'r', 'a', 'b', 'b', 'i', 't'] , while I am looking for ['c', 'a', 't', 'd', 'o', 'g', 'r', 'b', 'i'] .

How do I modify the list comprehension to remove duplicates?

Answer 1

Do you care about maintaining order?

>>> wordlist = ['cat','dog','rabbit']
>>> set(''.join(wordlist))
{'o', 'i', 'g', 'd', 'c', 'b', 'a', 't', 'r'}

Answer 2

Two approaches:

Preserving order:

>>> from itertools import chain
>>> from collections import OrderedDict
>>> list(OrderedDict.fromkeys(chain.from_iterable(wordlist)))
['c', 'a', 't', 'd', 'o', 'g', 'r', 'b', 'i']

If you're not fussed about order:

>>> list(set().union(*wordlist))
['a', 'c', 'b', 'd', 'g', 'i', 'o', 'r', 't']

Neither of this are using list-comps for side effects, eg:

[[letterlist.append(x) for x in y] for y in wordlist]

Is building a list of lists of Nones purely to mutate letterlist

Answer 3

While all other answers don't maintain order, this code does:

from collections import OrderedDict
letterlist = list(OrderedDict.fromkeys(letterlist))

See also, an article about several ways with benchmarks: Fastest way to uniqify a list in Python .

Answer 4

If you want to edit you own code:

[[letterlist.append(x) for x in y if x not in letterlist] for y in wordlist]

or

list(set([[letterlist.append(x) for x in y if x not in letterlist] for y in wordlist]))

else:

list(set(''.join(wordlist)))

Answer 5

You can use set to remove the duplicates but the order is not maintained.

>>> letterlist = list({x for y in wordlist for x in y})
>>> letterlist
['a', 'c', 'b', 'd', 'g', 'i', 'o', 'r', 't']
>>> 

Answer 6

wordlist = ['cat','dog','rabbit']
s = set()
[[s.add(x) for x in y] for y in wordlist]