How to launch, monitor and kill a process in Python

Question

I need to be able to launch a long-running process in Python. While the process is running, I need to pipe the output to my Python app to display it in the UI. The UI needs to also be able to kill the process.

I've done a lot of research. But I haven't found a way to do all three of those things.

subprocess.popen() lets me start a process and kill it if I need to. But it doesn't allow me to view its output until the process has finished. And the process I am monitoring never finishes on its own.

os.popen() lets me start a process and monitor its output as it is running. But I don't know of a way to kill it. I'm usually in the middle of a readline() call.

When using os.popen(), is there a way to know if there is any data in the buffer before calling read() or readline? For example...

output = os.popen(command)
while True:
    # Is there a way to check to see if there is any data available
    # before I make this blocking call?  Or is there a way to do a 
    # non-blocking read?
    line = output.readline()
    print(line)

Thanks in advance.

Answer 1

I'd recommend subprocess.Popen for fine-grained control over processes.

import subprocess


def main():
    try:
        cmd = ['ping', '8.8.8.8']
        process = subprocess.Popen(
            cmd,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
            universal_newlines=True,
            bufsize=1,
            text=True
        )
        while True:
            print(process.stdout.readline().strip())

    except KeyboardInterrupt:
        print('stopping process...')
        process.kill()


if __name__ == '__main__':
    main()

• Setting stdout and stderr kwargs to subprocess.PIPE allows you to read the respective streams via .communicate instead of having them be printed to the parent's streams (so they'd appear in the terminal you ran the script in)
• .kill() allows you to terminate the process whenever you want
• process.stdout and process.stderr can be queried at any time to obtain their current line, via readline() , or an arbitrary amount of buffer content via read() or readlines()