Python: subprocess.Popen() returns None

Question

I need to execute a CLI binary with args, keep the process alive and run multiple commands throughout the python script. So I am using Python and subprocess.Popen() in the following way:

from subprocess import Popen, PIPE

cmd = ["/full/path/to/binary","--arg1"]
process = Popen(cmd,stdin=PIPE, stdout=None)
process.stdin.write(f"command-for-the-CLI-tool".encode())
process.stdin.flush()

However, no matter how I call Popen(), the returned process object is None .

If I run process = Popen(cmd) , without specifying stdin and stdout , I can see the process running correctly in the output console, meaning that the binary path and args are correct, but the process object is still None , meaning that I cannot issue other commands afterwards.

EDIT: The point of this is that I want to execute the following:

command = (
        f"cat << EOF | {cmd}\n"
        f"use {dbname};\n"
        "set optimizer_switch='hypergraph_optimizer=on';\n"
        f"SET forced_plan='{forced_plan}';\n"
        f"{query_text}\n"
        "EOF"
    )
    runtimes = []
    for _ in trange(runs):
        start = time.time()
        subprocess.run(command, shell=True, stdout=sys.stdout)
        runtimes.append(time.time() - start)

But this clearly measures the time of all the commands, whereas I am only interested in measuring the "query_text" command. This is why I am looking for a solution where I can send the commands separately and time only the one I am interested in. If I use multiple subprocess.run() , then the process instances will be different. I want the instance to be the same because the query depends on the previous commands.

Answer 1

With subprocess.run you can pass the entire input as... input .

    command = f"""\
use {dbname};
set optimizer_switch='hypergraph_optimizer=on';
SET forced_plan='{forced_plan}';
{query_text}
"""
    runtimes = []
    for _ in trange(runs):
        start = time.time()
        subprocess.run([cmd], text=true, input=command, stdout=sys.stdout)
        runtimes.append(time.time() - start)

I took out shell=True ; perhaps see also Actual meaning of shell=True in subprocess as well as perhaps Running Bash commands in Python which elaborates on several of the changes here.

Answer 2

Try using subprocess.run() instead of subprocess.Popen()

If you still use subprocess.Popen() , then you can use the .poll() method

But subprocess.Popen() will always return None if the execution of the command has not yet completed, or an exit code if the command has finished its execution.