How can I run a bash executable from a python script with the same i/o?

Question

So, what I'm trying to do is execute a program from a python script, which I execute in the shell. I've read other questions so I can do it with:

def bash(cmd):
    subprocess.Popen(cmd, shell=True, executable='/bin/bash')
bash('./my/file')

Now the first problem with this is I often want to terminate the program from shell. Ordinarily, if I did "./my/file" from terminal I could stop it with ctrl+c. But using subprocess runs it in the background somehow and I can only kill it through the command "top", and killing it by pid in bash. But the program spawns a large number of processes so fast I literally just can't kill it this way in a reasonable way.

Also, the second thing I want to do is to wait for the program to finish running before executing more of my python script. I tried setting...

process=subprocess.Popen(cmd, shell=True, executable='/bin/bash')
process.wait()

But that actually stops the program almost immediately after it starts.

I could just execute the scripts separately, but I don't see a way to use the results from the program without creating a third script that I have to run myself after the original python script and then the program are complete. Any suggestions?

Answer 1

For you first question, try preexec_fn argument:

import signal
import ctypes
libc = ctypes.CDLL("libc.so.6")
def set_pdeathsig(sig = signal.SIGTERM):
    def callable():
        return libc.prctl(1, sig)
    return callable
p = subprocess.Popen(args, preexec_fn = set_pdeathsig(signal.SIGTERM))

This will ensure all the child processes spawned by subprocess.Popen will automatically exit right after the parent process exits. So all you need to kill is just the python process.

For you second question, try communicate method:

process=subprocess.Popen(cmd, shell=True, executable='/bin/bash', stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()

It will build a pipe between your python process and its child process through which input data can be passed to the child process and the result can be returned back. If you don't call the communicate method, generally the process will be running in the background, and the result if exists will be discarded.

By calling communicate method, parent process will block here, waiting for the child process to finish. out and err represent the normal output and the error output of the child process. You can use it to get the output of the bash script.