Generator vs List Append in Python

Question

For simple list creations, it is recommended to use list comprehension. However, when it comes to more complicated scenarios, spanning the list creation across multiple lines is more advisable.

eg for a simple task

[x for x in range(10)]

eg for a more complicated task

result = []
for x in complicated:
    if y is complex:
        for z in intricate_stuff:
             result.append(convoluted(z))

(I probably would still use a multi-line list comprehension for the above example, but anyways you get the idea.)

Recently, I found myself achieve list creation like that using a generator like:

def gen():
    for x in complicated:
        if y is complex:
            for z in intricate_stuff:
                 yield convoluted(z)

result = [x for x in gen()]

It's more natural for me to use a generator like this to create a list (well, generate a list), but I feel it might add unnecessary complexity.

Is this recommended? or, which way is more pythonic?

Answer 1

Generators builds the elements on the go, one at a time. So for statements like:

for e in my_generator():
    pass

Value of variable e is generated each time during iteration, the value of e in the previous iteration is not stored. Hence it doesn't store the entire elements of the list. So it is memory efficient.

But if you are planning on using generator inside a list comprehension, there is no point in using generator as all the elements need to be generated and stored.

In that case you can use list comprehension like below:

result = [ convoluted(z) for x in complicated if y is complex for z in intricate_stuff ]