Executing bash with subprocess.Popen

Question

I'm trying to write a wrapper for a bash session using python. The first thing I did was just try to spawn a bash process, and then try to read its output. like this:

from subprocess import Popen, PIPE
bash = Popen("bash", stdin = PIPE, stdout = PIPE, stderr = PIPE)
prompt = bash.stdout.read()
bash.stdin.write("ls\n")
ls_output = bash.stdout.read()

But this does not work. First, reading from bash's stdout after creating the process fails, and when I try to write to stdin, I get a broken pipe error. What am I doing wrong?

Just to clarify again, I'm not interested in running a single command via bash and then retrieving its output, I want to have a bash session running in some process with which I can communicate via pipes.

Answer 1

This works:

import subprocess
command = "ls"
p = subprocess.Popen(command, shell=True, bufsize=0, stdout=subprocess.PIPE, universal_newlines=True)
p.wait()
output = p.stdout.read()
p.stdout.close()

Answer 2

You can refer to my another answer: https://stackoverflow.com/a/43012138/3555925 , which use pseudo-terminal and select in handle stdin/stdout.

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import os
import sys
import select
import termios
import tty
import pty
from subprocess import Popen

command = 'bash'
# command = 'docker run -it --rm centos /bin/bash'.split()

# save original tty setting then set it to raw mode
old_tty = termios.tcgetattr(sys.stdin)
tty.setraw(sys.stdin.fileno())

# open pseudo-terminal to interact with subprocess
master_fd, slave_fd = pty.openpty()

# use os.setsid() make it run in a new process group, or bash job control will not be enabled
p = Popen(command,
          preexec_fn=os.setsid,
          stdin=slave_fd,
          stdout=slave_fd,
          stderr=slave_fd,
          universal_newlines=True)

while p.poll() is None:
    r, w, e = select.select([sys.stdin, master_fd], [], [])
    if sys.stdin in r:
        d = os.read(sys.stdin.fileno(), 10240)
        os.write(master_fd, d)
    elif master_fd in r:
        o = os.read(master_fd, 10240)
        if o:
            os.write(sys.stdout.fileno(), o)

# restore tty settings back
termios.tcsetattr(sys.stdin, termios.TCSADRAIN, old_tty)

Answer 3

The read() call is blocking, so it reads the output of the first command and hangs because further data could still arrive (ie. bash is still running). To perform a non-blocking read, see Non-blocking read on a subprocess.PIPE in python .

Answer 4

为什么不直接使用cmd 模块/raw_input 和 subprocess.Popen(shell = True)？