How to use reduce with list of lists

Question

What is wrong with the following?:

lss = reduce(list.extend, [[1],[2]], [])

causes:

Traceback (most recent call last):
  File "<pyshell#230>", line 1, in <module>
    lss = reduce(list.extend, [[1],[2]], [])
TypeError: descriptor 'extend' requires a 'list' object but received a 'NoneType'

I'm not sure where the NoneType is coming from.

Answer 1

Try this instead:

lss = reduce(lambda acc, ele : acc + ele, [[1],[2]], [])

lss
> [1, 2]

The problem is that extend() returns None (that's where the NoneType is coming from), and that won't work with what you want to do - the function passed to reduce() must return a value: the accumulated result so far.

Answer 2

I think it worth noting that:

sum([[1],[2]], [])

Will also work, and I'm pretty sure will be faster then passing a lambda to reduce.

I was curious as to the speed of different methods, so I did some testing:

reduce(lambda a,b:a+b, x, [])               3644.38161492
reduce(list.__add__, x, [])                 3609.44079709
sum(x,[])                                   3526.84987307
y = [];for z in x: y.extend(z)               143.370306969
y = [];map(y.extend,x)                        71.7020270824
y = [None]*400;del y[:];map(y.extend,x)       66.2245891094
list(itertools.chain(*x))                    102.285979986
list(itertools.chain.from_iterable(x))        96.6231369972
[a for b in x for a in b]                    203.764872074

And on PyPy (Because, why not)

reduce(lambda a,b:a+b, x, [])               4797.5895648
reduce(list.__add__, x, [])                 4794.01214004
sum(x,[])                                   4748.02929902
y = [];for z in x: y.extend(z)                56.9253079891
y = [];map(y.extend,x)                        73.8642170429
y = [None]*400;del y[:];map(y.extend,x)      152.157783031
list(itertools.chain(*x))                    633.854824066
list(itertools.chain.from_iterable(x))       629.917827129
[a for b in x for a in b]                     89.6922459602

x = [[1,2,3,4],[2,3,4,5],[3,4,5,6],[4,5,6,7],[5,6,7,8],[6,7,8,9],[7,8,9,10],[8,9,10,11]]*100

Conclusions:

1. Using a lambda in your reduce is slow
2. The specialized sum function is faster then reduce
3. Adding lists is slow.
4. Python loop overhead is significant.

Answer 3

As noted by Óscar López, list.extend returns None and thus cannot be used with reduce . Alternatively to the suggested use of the lambda function, you could also use list.__add__ with reduce :

>>> reduce(list.__add__, [[1],[2]], [])
[1, 2]

Answer 4

You may want to use itertools.chain

Make an iterator that returns elements from the first iterable until it is exhausted, then proceeds to the next iterable, until all of the iterables are exhausted. Used for treating consecutive sequences as a single sequence. Equivalent to:

def chain(*iterables):
    # chain('ABC', 'DEF') --> A B C D E F
    for it in iterables:
        for element in it:
            yield element