Why do some Unix commands not work when called from inside Python? (command not found)

Question

I often wish to perform Unix commands from inside Python, but I have found recently that some commands are not found. An example is the 'limit' command:

$ echo $SHELL
/bin/tcsh
$ limit vmemoryuse 1000m
$ python
Python 2.7.3 (default, Aug  3 2012, 20:09:51) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-50)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.system("echo $SHELL")
/bin/tcsh
0
>>> os.system("limit vmemoryuse 1000m")
sh: limit: command not found
32512
>>>

Another example is the 'setenv' command. Why do these commands do not work inside Python? I have tried using both the 'os' and 'subprocess' modules without success. Does anybody know of another module or method that will allow me to successfully call these commands from inside Python?

Answer 1

That's because some shell commands are not really programs, but internal shell commands.

The classical example is cd : if it were an external program it would change the current directory of the new process, not the one of the shell, so it cannot be an external program.

Roughly speaking there are two types of internal shell commands:

1. Commands that are implemented by the shell of efficiency's sake, but it still exists as an standalone program: true , false , test , sleep ...
2. Commands that change the environment of the shell, and so cannot be done from a child process: cd , umask , setenv , ulimit ...

The commands in the first category are quite shell specific. The commands in the second category, not so much.

For details see the man page of the relevant shell ( man bash for example).

And if you want to know about an specific command run:

$ type -a <command>

Type is a bashism , I don't know the equivalent in tcsh, but which is an external program, so this:

$ which -a <command>

will show you whether your command exists as an external program, but it knows nothing about shell internals.

If you need the functionality of an internal command (of type 2 above) in your Python program you need to use the relevant system call. Hopefully it will already be available in some module. If not, you would need to write your own wrapper in C.

About your specific commands:

• The environment ( setenv and getenv ) can be manipulated with os.environ or os.getenv , os.putenv , etc.
• For the process limits ( limit ) take a look at the resource module.