Python subprocess.check_output output differ from bash output of same command

Question

I'm trying to get the current bash version from python using subprocess.

In my terminal, I can use printf "%s\n" $SHELL to get /bin/bash and printf "%s\n" $BASH_VERSION to get 5.1.8(1)-release .

So I prepared this python script:

>>> import subprocess
>>> subprocess.check_output(['printf "%s\n" $SHELL'], shell=True, stderr=subprocess.STDOUT)
b'/bin/bash\n'
>>> subprocess.check_output(['printf "%s\n" $BASH_VERSION'], shell=True, stderr=subprocess.STDOUT)
b'\n' <-- This output is wrong

Anyone could explain me why the output of printf $BASH_VERSION in subprocess differ from the terminal one?

Answer 1

BASH_VERSION is an internal variable to bash shell, it simple doesn't exists outside of it. As per https://docs.python.org/3/library/subprocess.html#popen-constructor

On POSIX with shell=True, the shell defaults to /bin/sh

So if /bin/sh on your system is not an alias to bash - you won't get anything. To specify shell explicitly, you could either call it, or use executable as kwarg.

Some tests that i ran on my system, where /bin/sh linked to /usr/bin/bash

subprocess.check_output(['echo $BASH_VERSION'], shell=True, stderr=subprocess.STDOUT)                                                                                              
b'4.4.23(1)-release\n'

subprocess.check_output(['echo $BASH_VERSION'], shell=True, stderr=subprocess.STDOUT, executable='/bin/zsh')                                                                       
b'\n'

subprocess.check_output(['echo $BASH_VERSION'], shell=True, stderr=subprocess.STDOUT, executable='/bin/bash')                                                                      
b'4.4.23(1)-release\n'

Note that in second example i invoke zsh which doesn't have this variable

$SHELL variable is actually inherited by subshells from your actual shell, but it doesn't mean that command is actually ran using that SHELL.

In my case, my user has zsh as default shell, but running

bash -c "echo $SHELL"                                                                                                                        
/usr/bin/zsh

Will still return zsh, as you can see

Answer 2

$SHELL is misleading, and not telling you what you think it is. The second command (showing $BASH_VERSION ) is correct; subprocess.check_output() is running the command under /bin/sh, which is dash instead of bash in recent versions of Ubuntu (and many other linux distros), so BASH_VERSION is not set in that shell.

Contrary to popular belief, the SHELL variable does not indicate the currently executing shell. bash will generally set it to the current user's default shell . From the [bash manual section on "Bash Variables"]:

SHELL
This environment variable expands to the full pathname to the shell. If it is not set when the shell starts, Bash assigns to it the full pathname of the current user's login shell.

So if your default shell (as defined in someplace like /etc/passwd) is /bin/zsh, and you manually run a bash shell under that, it'll set SHELL to "/bin/zsh" even though you're actually running bash at the time.

Shells other than bash may or may not set SHELL to anything at all (note that it's listed in the "Bash Variables", not the "Bourne Shell Variables" section with the variables other Bourne-family shells use). dash and zsh don't set PATH at all, and ksh seems to do something else with it.

So, you may ask, if subprocess.check_output() runs under dash, why is it printing anything at all for $SHELL ? I suspect it's because it inherited that from your interactive bash shell. If you started a zsh shell and ran echo $SHELL in that, you'd presumably see "/bin/bash" there too.