Python3，懒惰的打印

Question

for my last project in Python3, i used a custom lazy generator to generate my data. Then use imap from a Pool (multiprocessing). So at this point, not any computation have been made. The next step is to output the computed data on a file. To do so, I either print(list(data)) or print(*data) which causes the computation of the whole data (approx 1.5Gib right now, bit will grow fast), either do a for loop and print each piece of data, which do a lot of call to print (approx 10e6 calls right now, but will grow fast).

So, is there a way to make print iterate over a lazy generator?

Thank you.

Answer 1

Using this recipe from the itertools docs:

def grouper(iterable, n, fillvalue=None):
    "Collect data into fixed-length chunks or blocks"
    # grouper('ABCDEFG', 3, 'x') --> ABC DEF Gxx
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)

You can batch the calls to print yourself:

for batch in grouper(data, 1000, ''):
    print('\n'.join(batch))