Pipe between python scripts

Question

I have a simple script that reads values from a device and outputs them via print , and another script, which listens on stdin and interprets each number. The device outputs one number each second. Surprisingly, piping the scripts on my ubuntu box does not work. However, if the first script is made not to read from the device but generate random numbers as fast as it can, the second script successfully receives the data.

Below is a simplified example of my situation.

print.py:

#!/usr/bin/env python2
import time
import sys

while True:
    time.sleep(1)  # without this everything works
    print "42"
    sys.stdout.flush()

read.py:

#!/usr/bin/env python2
import sys

while True:
    for str in sys.stdin:
        print str

Command line invocation:

vorac@laptop:~/test$ ./print.py | ./read.py

---

Here is the end result. The first script reads from the device and the second graphs the data in two separate time frames (what is shown are random numbers).

Answer 1

Ah, now that is a tricky problem. It happens because the iterator method for sys.stdin (which is xreadlines() ) is buffered. In other words, when your loop implicitly calls next(sys.stdin) to get the next line of input, Python tries to read from the real under-the-hood standard input stream until its internal buffer is full, and only once the buffer is full does it proceed through the body of the loop. The buffer size is 8 kilobytes , so this takes a while.

You can see this by decreasing the time delay in the sleep() call to 0.001 or some such value, depending on the capabilities of your system. If you hit the time just right, you'll see nothing for a few seconds, and then a whole block of 42 s come out all at once.

To fix it, use sys.stdin.readline() , which is unbuffered.

while True:
    line = sys.stdin.readline()
    print line

You might also want to strip off the trailing newline before printing it, otherwise you'll get double line breaks. Use line.rstrip('\\n') , or just print line, to suppress the extra newline that gets printed.

Answer 2

I changed your read.py and it worked for me :), you forget to .readline() from stdin .

import sys
while True:
    line = sys.stdin.readline()
    if line:
        print line.strip()
    else:
        continue

Output is :

$ python write.py | python read.py 
42
42
42
42
42
42
42