explain this python3 vs python behavior of sys.stdin

Question

I have to write a code that somehow works for both python versions and I can't understand the behavior of the code:

from __future__ import print_function
import sys
sys.stdout.flush()
print("input: ", end="")
f = sys.stdin.readline()
print(f)

When I run the code with python2 it behaves like I expect:

$ python2 test_input.py 
input: foo bar
foo bar

When I run the code with python3 it behaves strange. It first reads the input and then prints the prompt:

$ python3 test_input.py 
foo bar
input: foo bar

Can you explain this and suggest a fix?

Answer 1

You got your flush() and print() backwards. Here is the correct order:

# First, write to stdout
print("input: ", end="")
# Then, flush all data in the stdout buffer
sys.stdout.flush()

Answer 2

The difference is that CPython 2 uses C stdio to implement standard streams such as sys.stdin , sys.stdout (used by print() ) while Python 3 reimplements IO on top of system API eg, open, read, write on POSIX.

To avoid thinking about how print() is implemented, the same issue occurs if sys.stdout is used directly:

# __main__.py
import sys

sys.stdout.write("input: ")
f = sys.stdin.readline()
sys.stdout.write("*" + f)

On Python 3 "input: " is not printed before the readline() call:

$ python2 .
input: foo bar
*foo bar
$ python3 .
foo bar
input: *foo bar

In C, stdout is flushed before reading any input in the interactive case (it is undefined behavior if an output operation is followed by an input operation on the same update stream without fflush() in between). This C program prints "input: " before asking for input as expected:

#include <stdio.h>

#define MAXLEN 100

int main(void) {
  char buf[MAXLEN] = {0};

  fputs("input: ", stdout);
  if (!fgets(buf, MAXLEN, stdin))
    return 1;

  fputs(buf, stdout);
  return 0;
}

That is why the workaround: calling sys.stdout.flush() before sys.stdin.readline() suggested by @Dietrich Epp works.

It is a deficiency in Python 3 implementation. stdout shall be flushed by default before reading from stdin if both point to the same place ( os.path.samefile(sys.stdout.fileno(), sys.stdin.fileno()) eg, if both tty). You could report the issue at Python bug tracker .

Answer 3

In Python 3.3 the flush keyword was added to the print() function :

Changed in version 3.3 : Added the flush keyword argument.

In versions before 3.3, the print() function would always flush, even when using end='' . Your exact behaviour also reproduced using Python 3.2:

$ python3.2 test.py 
foo bar
input: foo bar

To get the same behaviour, flush in Python 3.3:

try:
    print("input: ", end="", flush=True)
except TypeError:
    print("input: ", end="")

or use sys.stdout.flush() after print() calls on Python 3.3 or up.