Executing python module through popen from a python script/shell

Question

I have a python module that is executed by the following command:

python3 -m moduleName args

Trying to execute it from a script using subprocess.popen.

subprocess.Popen(command, shell=True, text=True, stdout=subprocess.PIPE)

Based on subprocess documentation, we are recommended to pass a sequence rather than a string. So when I try to pass the below command as the argument

command = ['python3','-m','moduleName','args']

I end up getting a python shell instead of the module being executed. If I pass it as a string, things are working as expected. I'm not able to find documentation or references for this.

Can someone please help throw some light into this behavior? What would be the best way to make this work?

Thanks!

Answer 1

This behavior is caused by the shell=True option. When Popen runs in shell mode (under POSIX), the command is appended to the shell command after a "-c" option ( subprocess.py , Python 3.9 ):

args = [unix_shell, "-c"] + args

When the list of arguments is expanded, the first argument after '-c' (in your case, 'python3') is treated as the parameter to '-c'. The other arguments are interpreted as further arguments to the unix_shell command. The -m , for example, activates job control in bash, as outlined in the bash manual .

The solution is to either

• pass the command as a single string, as you did, or
• do not set the shell option for Popen , which is a good idea anyway, as it is lighter on resources and avoids pitfalls like the one you encountered.