input command doesn't seem to work when used with popen python

Question

I am writing a small python application which executes scala commands. A user can insert the command through the STDIN and then the python app forwards them to the scala interpreter. Once the command is executed, the app shows the result of the operation.

The idea is to use Popen to create a pipe by which I can send commands and read results. The idea is quite simple, but it doesn't work. What I don't understand is, why the sys.stdin doesn't work anymore after the pipe is opened. This makes impossible to read commands in python.

This is the code I am using:

import sys
from subprocess import Popen, PIPE

with Popen(["scala"], stdout=PIPE, stdin=PIPE, bufsize=0, universal_newlines=True) as scala:
    while True:
        print("Enter scala command >>> ", end="")
        sys.stdout.flush()
        command = input()
        scala.stdin.write(command)
        scala.stdin.flush()
        print(scala.stdout.readline())

Answer 1

You need to read all the lines from when the scala starts then input the command with a new line and get the two lines of output after:

from subprocess import Popen, PIPE

with Popen(["scala"], stdout=PIPE, stdin=PIPE, bufsize=0, universal_newlines=True) as scala:
    for line in scala.stdout:
        print(line)
        if not line.strip():
            break
    while True:
        command = input("Enter scala command >>> \n")
        scala.stdin.write(command+"\n")
        scala.stdin.flush()
        for line in scala.stdout:
            if not line.strip():
                break
            print(line)

An example run:

Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60).

Type in expressions to have them evaluated.

Type :help for more information.



Enter scala command >>> 3+4
scala> 3+4

res0: Int = 7

Enter scala command >>> 4 * 4
scala> 4 * 4

res1: Int = 16

Enter scala command >>> 16 / 4
scala> 16 / 4

res2: Int = 4

To get it to work from bash running it with unbuffer seems to sort out the output issues:

from subprocess import Popen, PIPE

with Popen(["unbuffer", "-p","scala"], stdout=PIPE, stdin=PIPE, bufsize=0, universal_newlines=True) as scala:
    for line in scala.stdout:
        print(line)
        if not line.strip():
            break
    while True:
        command = input("Enter scala command >>> ")
        scala.stdin.write(command+"\n")
        scala.stdout.flush()
        for line in scala.stdout:
            if not line.strip():
                break
            print(line)

If you are using Mac Os x, you should probably use :

with Popen(["script", "-q", "/dev/null", "scala"], stdout=PIPE, stdin=PIPE, bufsize=0, universal_newlines=True) as scala:

From bash:

        print(line)
## -- End pasted text --
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_60).

Type in expressions to have them evaluated.

Type :help for more information.



Enter scala command >>> 4 + 2
scala> 4 + 2

res0: Int = 6

Enter scala command >>> 4 * 12
scala> 4 * 12

res1: Int = 48

Enter scala command >>> 100 // 25
scala> 100 // 25

res2: Int = 100

Enter scala command >>> 

More info regarding shell buffer issues:

• http://www.pixelbeat.org/programming/stdio_buffering/
• https://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe