Python：两个列表列表的交集

Question

I have a list of lists A and a list of lists B where A and B have many identical sublists.

What is the best way to get the unique sublists out of B and into A ?

A = [['foo', 123], ['bar', np.array(range(10))], ['baz', 345]]
B = [['foo', 123], ['bar', np.array(range(10))], ['meow', 456]]

=> A = [['foo', 123], ['bar', np.array(range(10))], ['baz', 345], ['meow', 456]]

I tried:

A += [b for b in B if b not in A]

But this gives me a ValueError saying to use any() or all() . Do I really have to test element by element for every sublist in B across every sublist in A ?

ERROR: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Answer 1

Usually, you would use one of the many ways to uniquify a list or several lists either in order or not.

Here is a way to uniquify two list that does not maintain order:

>>> A=[1,3,5,'a','c',7]
>>> B=[1,2,3,'c','b','a',6]
>>> set(A+B)
set(['a', 1, 'c', 3, 5, 6, 7, 2, 'b'])

Here is a way that does maintain order:

>>> seen=set()
>>> [e for e in A+B if e not in seen and (seen.add(e) or True)]
[1, 3, 5, 'a', 'c', 7, 2, 'b', 6]

The problem is that all elements must be hashable to use these methods:

>>> set([np.array(range(10)), 22])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unhashable type: 'numpy.ndarray'

One way around this is to use the repr of each element:

>>> set([repr(e) for e in [np.array(range(10)), 22]])
set(['22', 'array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])'])

Or a use a frozenset :

>>> set(frozenset(e) for e in [np.array(range(10)), np.array(range(2))])
set([frozenset([0, 1]), frozenset([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])])

In your case, the frozenset approach does not work on a list of lists:

>>> set(frozenset(e) for e in [[np.array(range(10)), np.array(range(2))],[np.array(range(5))
]])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <genexpr>
TypeError: unhashable type: 'numpy.ndarray'

So you would need to use flattened lists.

If the repr of the sublist is definitive proof of its uniquity, you could do this:

from collections import OrderedDict
import numpy as np

A = [['foo', 123], ['bar', np.array(range(10))], ['baz', 345]]
B = [['foo', 123], ['bar', np.array(range(10))], ['meow', 456]]

seen=OrderedDict((repr(e),0) for e in B)

newA=[]
for e in A+B:
    key=repr(e)
    if key in seen:
        if seen[key]==0:
            newA.append(e)
            seen[key]=1
    else:
        seen[key]=1
        newA.append(e)

print newA
# [['foo', 123], ['bar', array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])], ['baz', 345], ['meow', 456]]

Since the repr function returns a string that can be used by eval function to recreate the list, that is pretty definitive test but I cannot say for absolutely sure. It depends on what is in your list.

For example, the repr of a lambda cannot recreate the lambda:

>>> repr(lambda x:x)
'<function <lambda> at 0x10710ec08>'

But the string value of '<function <lambda> at 0x10710ec08>' is still definitively unique because the 0x10710ec08 part is the address in memory of the lambda (in cPython anyways).

You could also do what I mentioned above -- use a flattened list in frozenset as a signature of what you have seen or not:

def flatten(LoL):
    for el in LoL:
        if isinstance(el, collections.Iterable) and not isinstance(el, basestring):
            for sub in flatten(el):
                yield sub
        else:
            yield el      
newA=[]    
seen=set()
for e in A+B:
    fset=frozenset(flatten(e))
    if fset not in seen:
        newA.append(e)
        seen.add(fset)

print newA        
# [['foo', 123], ['bar', array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])], ['baz', 345], ['meow', 456]]

So if you have odd objects that are both unhashable and weird, non-unique repr string objects in A and B -- you are out of luck. Given your example, one of these methods should work though.

Answer 2

You could do

import numpy as np

A = [['foo', 123], ['bar', np.array(range(10))], ['baz', 345]]
B = [['foo', 123], ['bar', np.array(range(10))], ['meow', 456]]

res = set().update(tuple(x) for x in A).update(tuple(x) for x in B)

except for the np.array items, which are unhashable... not sure what to do with those.