Python 2 vs 3: Replace list comprehensions with map()?

Question

Given the following test:

>>> import timeit
>>> timeit.timeit("[x + 'abc' for x in ['x', 'y', 'z']]")
>>> timeit.timeit("map(lambda x: x + 'abc', ['x', 'y', 'z'])")

With Python 2.7 and 3.4 (Debian 8/testing/jessie) I get the following numbers:

Python27 Python34
1.3s     0.5s      map()
0.6s     0.9s      list comprehension

Map improved significantly with Python 3, the list comprehension suffered badly.

Question: When porting code from Python 2 to Python 3, is it recommended to change list comprehensions to map()?

Answer 1

You are not testing correctly. In Python 3, map() returns an iterator, not a list. You are not actually iterating in your test, only testing the creation of the iterator.

You'll need to include iteration to see which approach is faster; you could use collections.deque() with a length of 0, this will iterate without producing a new list object:

import timeit
timeit.timeit("deque([x + 'abc' for x in ['x', 'y', 'z']], maxlen=0)",
              'from collections import deque')
timeit.timeit("deque(map(lambda x: x + 'abc', ['x', 'y', 'z']), maxlen=0)",
              'from collections import deque')

By applying the deque() to both you even out the score again.

Now list comprehensions win on both platforms:

Python27 Python34
1.91s     2.00s      map()
1.18s     1.85s      list comprehension

You should really use far larger input lists to properly test the differences; too much o

The reason list comprehensions slowed down on Python 3 is because they got their own proper scope, just like generator expressions and dict and set comprehensions do on both Python 2 and 3.

If your map function is entirely implemented in C (as opposed to a lambda, which pushes back to Python, map() could win:

>>> timeit.timeit("deque([m(i) for i in ['x', 'y', 'z']], maxlen=0)",
...               "from collections import deque; from operator import methodcaller; m = methodcaller('__add__', 'abc')")
2.3514049489967874
>>> timeit.timeit("deque(map(methodcaller('__add__', 'abc'), ['x', 'y', 'z']), maxlen=0)",
...               'from collections import deque; from operator import methodcaller')
1.7684289459939464

Here the methodcaller() object avoids calling back into Python code by calling the str.__add__ method for each object used.