python中的集合不区分大小写

Question

I have a list that is generated from multiple lists. This combined list contains names that are generated by end users. Therefore contain similar names, but with different upper/lower case characters. I want to filter out the names that contain same characters and just keep the first found in the original list.

As an example I have the following list:

L0 = ['A_B Cdef', 'A_B Cdef', 'A_B Cdef', 'A_B CdEF', 'A_B CDEF','a_B CdEF', 'A_b CDEF', 'GG_ooo', 'a1-23456']

if I run:

L1 = list(set(L0))

I get:

['a1-23456', 'A_B Cdef', 'A_B CdEF', 'A_B CDEF', 'a_B CdEF', 'A_b CDEF', 'GG_ooo']

I would like to keep just the first of the names that have same characters.

So my result is:

['a1-23456', 'A_B Cdef', 'GG_ooo']

If I use .lower() , .upper() I get the list, but the names are lower/upper cased.

I just want to eliminate "duplicates" without considering case sensitive approach.

Help greatly appreciated.

Thanks!

Answer 1

You can track the .lower() version of the values using a set and then append the original values to a new list if their .lower() version isn't already in the set:

s = set()
L = []
for x in L0:
  if x.lower() not in s:
      s.add(x.lower())
      L.append(x)

print(L)
# ['A_B Cdef', 'GG_ooo', 'a1-23456']

Answer 2

使用哈希代替，我认为您不能通过集合轻松实现这一点。

L0 = {value.lower(): value for value in L0[::-1]}.values()

Answer 3

You already have several good answers, and the code below is probably overkill for your use-case, but just for fun I created a simple case-insensitive mutable set class. Note that it keeps the first string that it finds rather than letting it get clobbered by later entries.

import collections.abc

class CasefoldSet(collections.abc.MutableSet):
    def __init__(self, iterable=None):
        self.elements = {} 
        if iterable is not None:
            for v in iterable:
                self.add(v)

    def __contains__(self, value):
        return value.casefold() in self.elements

    def add(self, value):
        key = value.casefold()
        if key not in self.elements:
            self.elements[key] = value

    def discard(self, value):
        key = value.casefold()
        if key in self.elements:
            del self.elements[key]

    def __len__(self):
        return len(self.elements)

    def __iter__(self):
        return iter(self.elements.values())

    def __repr__(self):
        return '{' + ', '.join(map(repr, self)) + '}'

# test

l0 = [
    'GG_ooo', 'A_B Cdef', 'A_B Cdef', 'A_B Cdef', 
    'A_B CdEF', 'A_B CDEF', 'a_B CdEF', 'A_b CDEF', 'a1-23456',
]

l1 = CasefoldSet(l0[:4])
print(l1)
l1 |= l0[4:]
print(l1)
l2 = {'a', 'b', 'A_B Cdef'} | l1
print(l2)
l3 = l2 & {'a', 'GG_ooo', 'a_B CdEF'}
print(l3)

output

{'GG_ooo', 'A_B Cdef'}
{'GG_ooo', 'A_B Cdef', 'a1-23456'}
{'GG_ooo', 'A_B Cdef', 'a1-23456', 'b', 'a'}
{'a_B CdEF', 'a', 'GG_ooo'}

This class inherits various useful methods from collections.abc.MutableSet , but to make it a full replacement for set it does need a few more methods. Note that it will raise AttributeError if you try to pass it non-string items .

Answer 4

If you want to play by the rules, the best solution I can think of is a bit messy, using sets to track which words have appeared;

seen_words = set()
L1 = []
for word in L0:
    if word.lower() not in seen_words:
        L1.append(word)
        seen_words.add(word.lower())

If you want to get a little hackier there is a more elegant solution, you can use a dictionary to track which words have already been seen, and it's an almost one-liner;

seen_words = {}
L1 = [seen_words.setdefault(word.lower(), word) 
      for word in L0 if word.lower() not in seen_words]
print(L1)

---

Both solutions outputs the same result;

['A_B Cdef', 'GG_ooo', 'a1-23456']