Need function that read large txt file line by line and if no more lines, open file and iterate again

Question

I have a large txt file and want to iterate over it. Need a function that return me line by line like generator and if no more lines, open file and again iterate over it.

file = open('names.txt', 'r')

def generator():
    for line in file:
        if line:
            return line.strip()
        else:
       ,,,,,,,, 
          

for i in range(100000000):
    a = generator()
    print(a)

Answer 1

If you only want to process small files, itertools.cycle as proposed by rdas is a nice option. The good point is that you could even use it on non seekable file objects, for example the ones produced from sockets.

But if you want to process large files with a little memory footprint, it is better to scan the file and rewind it:

def line_generator(file):
    while True:
        for line in file:
            yield line.strip()
        file.seek(0)

with open('names.txt') as file:
    for i, line in enumerate(line_generator(file)):
        if i >= 100000000:
            print(line)

You used 100000000 but if you only print the lines, I would suggest a much smaller number...

Answer 2

Something like this doesn't require that the whole file is read into memory:

def read_n(filepath, times=2):
    for _ in range(times):
        with open(filepath, "r") as fh:
            yield from fh

for line in read_n("file.txt"):
    print(line)

Answer 3

You could use itertools.cycle to create an infinite generator from the lines you read from the file.

from itertools import cycle


def file_loop(file_name):
    with open(file_name) as f:
        lines = f.readlines()
    return cycle(lines)


for line in file_loop('test.txt'):
    print(line)

Which would keep looping over all the lines indefinitely:

line1
line2
...
linen
line1
line2
...