Remove duplicates from a nested list

Question

I have a list

A = [['1'],['1','2'],['1','2','3','1','2'],['3','3','3']]

and I want to make my list to

A = [['1'],['1','2'],['1','2','3'],['3']]

ie I want to remove duplicate elements within the elements in a list ..

Answer 1

One-liner (If order doesn't matter) :

A = [['1'],['1','2'],['1','2','3','1','2'],['3','3','3']]
A = [list(set(a)) for a in A]
print(A) # => [['1'], ['2', '1'], ['3', '2', '1'], ['3']]

One-liner (If order matters) :

A = [['1'],['1','2'],['1','2','3','1','2'],['3','3','3']]
A = [sorted(set(a), key=a.index) for a in A]
print(A) # => [['1'], ['1', '2'], ['1', '2', '3'], ['3']]

Answer 2

A functional version, with functools :

>>> import functools
>>> A = [['1'],['1','2'],['1','2','3','1','2'],['3','3','3']]
>>> print ([functools.reduce(lambda result,x:result if x in result else result+[x], xs, []) for xs in A])
[['1'], ['1', '2'], ['1', '2', '3'], ['3']]

The lambda function adds an element to the result list only if that element is not present in the list. Not very efficient, but keeps the order of elements.

Also note that with Python 2, you don't need to import functools : reduce is a builtin function.

Answer 3

You can use a generator:

def remove_dups(l):
   for a in l:
      new_l = []
      for b in a:
          if b not in new_l:
             new_l.append(b)
      yield new_l

A = [['1'],['1','2'],['1','2','3','1','2'],['3','3','3']]
print(list(remove_dups(A)))

Output:

[['1'], ['1', '2'], ['1', '2', '3'], ['3']]