如何从python中的列表中删除重复项

Question

I have a simple code here:

a=[['bcn09','113','shift1'],['bcn09','113','shift1'],['bps01','132','shift2']]

b=[]
for i in range (len(a)):
   if a[i] not in b:
      b.append([a[i]])


print (b)

The output i get is

b=[['bcn09','113','shift1'],['bcn09','113','shift1'],['bps01','132','shift2']]

ie the same as a The output i need is

b=[['bcn09','113','shift1'],['bps01','132','shift2']]

What am i doing wrong?

Thanks in advance

Answer 1

Very close! You could try this approach, hopefully it makes sense:

a=[['bcn09','113','shift1'],['bcn09','113','shift1'],['bps01','132','shift2']]

noDups = []
for i in a:
  if i not in noDups:
    noDups.append(i)
print(noDups)

Output:

>>>[['bcn09', '113', 'shift1'], ['bps01', '132', 'shift2']]

Answer 2

There is no need to create a new list when appending to 'b'.

Modify the below line from -

b.append([a[i]])

to

b.append(a[i])

The new output is (which you want) -

[['bcn09', '113', 'shift1'], ['bps01', '132', 'shift2']]

This explains more about the 'in' membership test operator.

Answer 3

For the most complete answer to getting only unique values from an iterable, itertools gives a recipe (which can also be found in the more-itertools package on PyPI ) for this:

def unique_everseen(iterable, key=None):
    "List unique elements, preserving order. Remember all elements ever seen."
    # unique_everseen('AAAABBBCCDAABBB') --> A B C D
    # unique_everseen('ABBCcAD', str.lower) --> A B C D
    seen = set()
    seen_add = seen.add
    if key is None:
        for element in filterfalse(seen.__contains__, iterable):
            seen_add(element)
            yield element
    else:
        for element in iterable:
            k = key(element)
            if k not in seen:
                seen_add(k)
                yield element

This implementation is tested, optimised, maintains order, is a generator so it will work on infinite iterables or cases where you need to be lazy, and allows you to use a key function.

>>> a=[['bcn09','113','shift1'],['bcn09','113','shift1'],['bps01','132','shift2']]
>>> list(unique_everseen(tuple(item) for item in a))
[('bcn09', '113', 'shift1'), ('bps01', '132', 'shift2')]

Note the change to tuples so the elements are hashable and can be added to the set. You can of course reverse this at the end if needed, although in most cases I can imagine tuples will probably be fine. (In the same way, I am creating a list from the generator to show the output, but most of the time you should be able to just work with the generator directly).

Answer 4

you can achive by using itertools

import itertools
a = [['bcn09','113','shift1'],['bcn09','113','shift1'],['bps01','132','shift2']]

a.sort()
new_num = list(a for a,_ in itertools.groupby(a))
print("New List", new_num)