如何合并两个数组列表（x，y，z，m），排除仅基于（x，y，z）的重复项

Question

I have two lists of the form

list1 = list(zip(SGXm, SGYm, SGZm, Lm))
list2 = list(zip(SGXmm, SGYmm, SGZmm, Lmm))

I want to merge them, while excluding duplicate (x,y,z) entries, and ignoring differences in L.

list1.extend(x for x in list2 if x not in list1)

Does the job only for my x,y,z, but I want to retain the Ls (of the first list when there is a choice).

Answer 1

You'll have to extract the triples you need for comparison.

seen = set(item[:3] for item in list1)
list1.extend(item for item in list2 if item[:3] not in seen)

Answer 2

If you want sorted output (particularly if you already have sorted input) itertools.groupby and heapq.merge combine nicely for this. If the inputs aren't already sorted, you'll need to do so. Either concatenate and sort all at once:

from operator import itemgetter

commonkey = itemgetter(0, 1, 2)
combined = sorted(list1 + list2, key=commonkey)

or if they're already sorted, or you want to sort independently, you use heapq.merge and avoid making shallow copies of the inputs:

# Explicit sort calls only necessary if inputs not already sorted
list1.sort(key=commonkey)
list2.sort(key=commonkey)

# Combine already sorted inputs with heapq.merge, avoiding intermediate lists
combined = heapq.merge(list1, list2, key=commonkey)

Whichever approach you chose, you then follow it up with a simple comprehension over groupby to keep only one copy of each unique key by just getting the first entry in each unique group:

# Groups neighboring entries with the same key, and we keep only the first one
uniq = [next(g) for _, g in itertools.groupby(combined, key=commonkey)]