python list comprehension on generator with if condition

Question

I am trying to perform list comprehension with if condition on a generator, but the code gets stuck in for loop.

import time
def gen():
    a = 0
    b = 1
    while True:
        a += 1
        b += 1
        yield a, b
    init_time = time.time() 
    m = [{'a': x, 'b': y} for x, y in gen() if time.time() - init_time < 3]

I realised this is because gen() will return infinitely and for will continue to execute till gen() is iterable can there be another way?

Answer 1

The if condition in a list comprehension is not a stop condition, it's a filter . As written, after 3 seconds it will simply start ignoring the pairs coming from the generator, and will never return the list it's made so far.

Another problem is that the list comprehension is currently inside the generator, which is (except in very specific circumstances) not how one uses list comprehensions with generators. A list comprehension iterates over the objects produced by the generator independent of its definition.

Finally, your generator is infinite . While infinite generators are perfectly valid and very useful, they cannot be passed to list comprehensions, because the list comprehension wants to consume the entire generator, which is by definition impossible with an infinite one. However, one can write a finite generator that adapts an infinite one, stopping when a condition is reached. itertools.islice is an example of such a wrapper in a standard library, but you can easily write your own. A time-based wrapper might look like this:

def iter_until(tm, iterable):
    t0 = time.time()
    for val in iterable:
        yield val
        if time.time() - t0 > tm:
            break

This wrapper can be easily combined with the original infinite generator and used in list comparehensions:

def gen():
    a = 0
    b = 1
    while True:
        a += 1
        b += 1
        yield a, b

m = [{'a': x, 'b': y} for x, y in iter_until(3, gen())]