sys.stdin.read() in subprocess never returning

Question

I'm trying to understand subprocess.Popen and subprocess.PIPE. While doing so, I created three small scripts.

The goal of this exercise is, to print data with one script, read it from the other and echo this back to the first script.

The problem is, that the last sys.stdin.read() prevents the entire code from working properly.

test.py

#!/usr/bin/python3

from subprocess import Popen, PIPE

proc1 = Popen(["python3","script1.py"], stdin=PIPE, stdout=PIPE)
proc2 = Popen(["python3","script2.py"], stdin=PIPE, stdout=PIPE)

first_output = proc1.stdout.read()

print("[test.py:] " + str(first_output))

proc2.stdin.write(first_output)
proc2.stdin.flush()
proc2.stdin.close()

answer = proc2.stdout.read()

print("[test.py:] " + str(answer))

proc1.stdin.write(answer)

script1.py

import sys

sys.stdout.write("Hello world")
sys.stdout.flush()

answer = str(sys.stdin.read())

The last line in script1.py answer = str(sys.stdin.read()) causes the entire program to get stuck. If I comment it out, everything works fine.

Why is that so and why am I not able to communicate any further? I didn't find an answer yet.

script2.py

import sys

input_read = str(sys.stdin.read())

sys.stdout.write("[script2.py:] " + input_read + " input read")
sys.stdout.flush()

Answer 1

When you do:

first_output = proc1.stdout.read()

this tries to read everything proc1 has to say; that is, it reads until the file handle is closed. That doesn't happen, because proc1 is itself waiting to read:

answer = str(sys.stdin.read())

Communicating between processes using pipes can be a bit tricky. I'd recommend reading through the documentation for the subprocess module .

To solve your immediate problem, I know of two simple solutions. One would be to switch to communicating in lines. In script1.py , write a newline:

sys.stdout.write("Hello world\n")
sys.stdouf.flush()

answer = str(sys.stdin.readline())

And in script2.py add a newline and switch to readline as well:

input_read = str(sys.stdin.readline())

sys.stdout.write("[script2.py:] " + input_read + " input read\n")
sys.stdout.flush()

The other approach would be to switch to using read1 instead of read. It takes an argument for the number of bytes to read, but doesn't wait to receive that much data before returning it.