在python中制作唯一列表

Question

What is the most pythonic way of making a list unique using custom equality operator?

For instance you have a list of dicts L , and you want a new list M such that for all dicts d , e in M and one specific x

d[x] != e[x]

How can this be done?

Answer 1

In your case (and all cases where equivalence boils down to the equivalence of some kind of key), you can simply construct a dictionary, where the keys are the values you want to compare:

L = [{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}, {'key': 'foo', 'v': 44}]
x = 'key'
M = {d[x]:d for d in L}.values()
# In old Python versions: dict((d[x],d for d in L)).values()

Note that the result is not deterministic, both

[{'key': 'foo', 'v': 44}, {'key': 'bar', 'v': 43}]

and

[{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}]

are valid results.

In the general case, simply check all accepted values:

def unique(iterable, is_eq):
  tmp = []
  for el in iterable:
    if not any(is_eq(inTmp, el) for inTmp in tmp):
      tmp.append(is_eq)
  return tmp

Note that this means that your comparison function will be called O(n²) times instead of n times.

Answer 2

Based on FUD's comment to phihag. Note that key function must return a hashable value.

def unique(iterable, key=lambda x : x):
    seen = set()
    res = []
    for item in iterable:
        k = key(item)
        if k not in seen:
            res.append(item)
            seen.add(k)
    return res

from operator import itemgetter
L = [{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}, {'key': 'foo', 'v': 44}]
print unique(L, key=itemgetter('key'))
#[{'key': 'foo', 'v': 42}, {'key': 'bar', 'v': 43}]

Answer 3

I'm not sure this sort of thing admits a one-liner, but it seems to me that the set class is the key to what you want.

M = []
uniques = set(d[x] for d in L)
for d in L:
    if d[x] in uniques:
        uniques.remove(d[x])
        M.append(d)

Note: phihag's answer seems more Pythonic, but this might be a bit more self-documenting

Answer 4

Using dictionary comprehension:

def unique(itrable,key):
    return {key(x):x for x in itrable}.values()

>>> unique('abcdbbcdab', lambda x: x)
['a', 'c', 'b', 'd']

>>> unique([10, -20, 20, 30], lambda x: abs(x))
[10, 20, 30]