Output from subprocess is not available on unbuffered stdout pipe before the subprocess terminates?

Question

I've created a subprocess using subprocess.Popen(['myapp'], stdin=PIPE, stdout=PIPE, stderr=PIPE, bufsize=0) that executes a C-program that writes to stdout using eg puts() .

The problem is that the Python program blocks in p.stdout.read(1024) , although the subprocess starts by puts("HelloWorld") . Only after the subprocess terminates, is output available on p.stdout . I thought that bufsize=0 would mean that pipes become unbuffered, so that output is immediately available on the pipe.

I have read the below question, which states that newlines should cause output to be flushed. However, puts() prints a newline, so are the pipes not recognized as an interactive device?

Difference between puts() and printf() in C while using sleep()

It's because puts is also outputting a newline character which, on devices that can be determined to be interactive, causes flushing by default (for standard output) (a).

Any ideas?

Answer 1

This is application behavior. Even if the pipe is unbuffered, applications normally buffer information that they are going to write to a file (any type of file) for some time before actually writing it. As Jon's comment above indicates, a system-call like fflush() can be used by programs to ensure that they actually have posted the data, and, if applicable, that a physical I/O operation has actually completed.