Python generators: Errors visible only after commenting

Question

I was trying following python code to simulate 'tail' command of *nix systems.

import sys
def tail(f):
    print 'in tail with ',f
    f.seek(0,2)
    while True:
        line = f.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

if(len(sys.argv) >= 2):
    print 'calling tail'
    tail(open(sys.argv[1],'r'))
else:
    print 'Give file path.\n'

I did an error (missed importing time module). However, what's odd is that no error was getting thrown and program was quitting silently. Output (before commenting):

$ python tail.py /var/log/dmesg 
calling tail

However if I comment lines following the one using the time module, the error does get thrown.

import sys
def tail(f):
    print 'in tail with ',f
    f.seek(0,2)
    while True:
        line = f.readline()
        if not line:
            time.sleep(0.1)
        #     continue
        # yield line

if(len(sys.argv) >= 2):
    print 'calling tail'
    tail(open(sys.argv[1],'r'))
else:
    print 'Give file path.\n'

Output (after commenting)

$ python tail.py /var/log/dmesg 
calling tail
in tail with  <open file '/var/log/dmesg', mode 'r' at 0x7fc8fcf1e5d0>
Traceback (most recent call last):
  File "tail.py", line 14, in <module>
    tail(open(sys.argv[1],'r'))
  File "tail.py", line 8, in tail
    time.sleep(0.1)
NameError: global name 'time' is not defined

Can anyone please explain why the error was not getting thrown in case one (before commenting)? Shouldn't a error be thrown as soon as interpreter comes on that line?

Corrected program:

import sys
import time
def tail(f):
    print 'in tail with ',f
    f.seek(0,2)
    while True:
        line = f.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line

if(len(sys.argv) >= 2):
    print 'calling tail'
    t = tail(open(sys.argv[1],'r'))
    for i in t:
        print i
else:
    print 'Give file path.\n'

Output:

$ python tail.py hello.txt 
calling tail
in tail with  <open file 'hello.txt', mode 'r' at 0x7fac576b95d0>
hello there 1

hello there 2

hello there 3

Thanks for the responses.

Answer 1

Short Answer

First one is instantiating a generator (but not assigning it to a variable) and second one is a function call.

---

Long Answer

This is because of dynamic type checking of python, when you have the yield statement, your function behaves as a generator and this line -

tail(open(sys.argv[1],'r'))

means that you are instantiating the generator not calling a function . You'll get that error when you assign this instance to some variable and call the next method for generator which actually fires it up ie -

t = tail(open(sys.argv[1],'r')) # t is a generator here 
t.next()

The other case in which you removed the yield statement, it started behaving as a normal function which means - tail(open(sys.argv[1],'r')) is a function call now, and hence it threw an error.

What I meant by dynamic is python doesn't check these kind of errors until it reaches that statement, which in first case wasn't.

Answer 2

With yield in the function, it is a generator. Generator functions only execute their contained code when the next value is requested. Simply calling a generator function merely creates that generator object. If you do so without doing anything with that object, such as looping through it, nothing will happen.

Removing the yield makes the function evaluate eagerly, so its code is actually executed.

If you actually iterated over the generator, it would produce an error if/when readline() produced an empty line. Since such an empty line can only occur at the end of a file (what look like blank lines actually contain a single linefeed character), putting it in a loop doesn't make sense anyway. Instead of this:

while True:
    line = f.readline()
    if not line:
        time.sleep(0.1)
        continue
    yield line

Use this:

for line in f:
    yield line

And instead of this:

if(len(sys.argv) >= 2):
    print 'calling tail'
    tail(open(sys.argv[1],'r'))

You should actually execute the generator's contents, with something like this:

if(len(sys.argv) >= 2):
    print 'calling tail'
    for line in tail(open(sys.argv[1],'r')):
        print line