如何为subprocess.call创建自定义输出流

Question

I am trying to get realtime output of a subprocess.call by defining my own output stream but it doesn't seem to work.

Reason: I want to run a subprocess and get output of that call to both stdout(in realtime so i can look at the script and see current progress) as well as logging it to a file

subprocess.py:

import time

while True:
    print("Things")
    time.sleep(1)

mainprocess.py

import subprocess
import io

class CustomIO(io.IOBase):
    def write(self, str):
        print("CustomIO: %s"%str)
        # logging to be implemented here

customio = CustomIO()
subprocess.call(["python3", "print_process.py"], stdout=customio)

But when i run this code i get this error message:

Traceback (most recent call last):                                   
    File "call_test.py", line 9, in <module>                           
        subprocess.call(["python3", "print_process.py"], stdout=customio)
    File "/usr/lib/python3.4/subprocess.py", line 537, in call         
        with Popen(*popenargs, **kwargs) as p:                           
    File "/usr/lib/python3.4/subprocess.py", line 823, in __init__     
        errread, errwrite) = self._get_handles(stdin, stdout, stderr)    
    File "/usr/lib/python3.4/subprocess.py", line 1302, in _get_handles
        c2pwrite = stdout.fileno()                                       
io.UnsupportedOperation: fileno

So, anyone have any clue if this is possible?

Am i inheriting the wrong baseclass?

Am i not overloading the proper methods?

Or am i completely off the rails and should be going about this in a completely different way?

Answer 1

If you want to process the output of a subprocess, you need to pass stdout=subprocess.PIPE . However, call() and run() will both wait until the process is finished before making it available, so you cannot handle it in real time using these functions.

You need to use subprocess.Popen :

import subprocess as sp

def handle_output(output_line):
    ...

my_process = sp.Popen(["python3", "print_process.py"],
                      stdout=sp.PIPE,
                      universal_newlines=True) # changes stdout from bytes to text

for line in my_process.stdout:
    handle_output(line)

my_process.wait()

Update: Make sure to flush the output buffer in your child process:

while True:
    print("Things", flush=True)
    time.sleep(1)

Answer 2

You need to specify and open stream with a file descriptor. fileno isn't implemented for io.IOBase because this is just an in-memory stream:

Frequently Used Arguments

stdin , stdout and stderr specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values are PIPE , DEVNULL , an existing file descriptor (a positive integer), an existing file object, and None . PIPE indicates that a new pipe to the child should be created. DEVNULL indicates that the special file os.devnull will be used. With the default settings of None , no redirection will occur;

So you might use sockets, pipes, and open files as stdout , the file descriptor is passed to the child process as it's stdout. I didn't use sockets with subprocess.Popen though, but I expect them to work, I believe what matters here is the file descriptor to the child, not what type of object the file descriptor points to.