Streaming stdin/stdout in Python

Question

I'm trying to stream a bash shell to/from a simple WebSockets UI, but I'm having trouble redirecting the IO. I want to start an instance of bash and connect stdout and stdin to write_message() and on_message() functions that interact with my web UI. Here's a simplified version of what I'm trying to do:

class Handler(WebSocketHandler):
    def open(self):
        print "New connection opened."
        self.app = subprocess.Popen(["/bin/bash", "--norc", "-i"], stdout=subprocess.PIPE, stdin=subprocess.PIPE, shell=False)
        thread.start_new_thread(self.io_loop, ())

    def on_message(self, message):
        self.app.stdin.write(message)

    def on_close(self):
        self.app.terminate()

    def io_loop(self):
        while self.app.poll() is None:
            line = self.app.stdout.readline()
            if line:
                self.write_message(line)

While bash appears to start and on_message does get called, I don't get any output. readline() remains blocking. I've tried stdout.read(), stdout.read(1), and various buffer modifications, but still no output. I've also tried hardcoding commands with a trailing '\\n' in on_message to isolate the issue, but I still don't get any output from readline().

Ideally I want to stream each byte written to stdout in realtime, without waiting for EOL or any other characters, but I'm having a hard time finding the right API. Any pointers would be appreciated.

Answer 1

It looks to me like the line:

line = self.app.stdout.readline()

will block the ioloop from running because the application will spend most of its time hung up in the readline() waiting for the application to write some output instead. To get this to work, you are going to have to get the stdin and stdout of the process (and what about stderr ? — you need to capture that too), switch them into non-blocking mode, and add them to the set of file descriptors that the ioloop spends its time looping on.