why are system calls so much slower in python compared to c++?

Question

i have 2 codes that are identical in the sense that they complete the same task. one code was written in python, the other in c++. all the codes do is call an executable (the executable generates an ascii file). in c++, i use the system() command to call the executable. in python, i have used many things including os.system subprocess.call subprocess.popen .

i realize that c++ is a compiled language while python is interpreted. and i also realize that the python calls have more overhead. but the c++ code does the job nearly 100 times faster than the python code. c++ time was about 0.004 seconds. python time was around 0.35 seconds.

even a simple pwd command takes more than 10 times longer with python than it does with c++. if the overhead is what is slowing the python code down, is there a faster option in python than what i have already tried?

here is a simple python code:

from os import system
from time import time

t0 = time();
system("pwd");
print "duration: ",time()-t0;

and here is the same thing in c++:

#include <iostream>
#include <sys/time.h>
double diff(timespec start, timespec end) { return (end.tv_nsec-start.tv_nsec)/1e9; }

int main()
{
    timespec t0, t1;
    clock_gettime(CLOCK_PROCESS_CPUTIME_ID, & t0);
    system("pwd");
    clock_gettime(CLOCK_PROCESS_CPUTIME_ID, & t1);

    std::cout << "duration: " << diff(t0,t1) << "\n";

    return 0;
}

i used gcc to compile the c++ code. you have to use the -lrt option to get the code to compile correctly.

you can run the code yourself. my timing methods could be wrong. but if they are okay, then the python script takes more than 10 times as long to execute the pwd command compared to the c++ executable

Answer 1

C 'exec' call executes the program directly.

While the Python 'system' call first executes bash, that executes the program in question.

Answer 2

You can use execvp directly in python

import os

binary = "ls"
options = [binary, "-l"]

newpid = os.fork()
if newpid == 0:
     # we are in the child process
     os.execvp(binary, options)
     os._exit(1)

os.wait()
print "executed", " ".join(options)

Answer 3

I cooked up a little script and execution time was much faster than what you are seeing.

td@timsworld2:~/tmp/so$ cat nothing.py
#!/usr/bin/env python
import subprocess
import sys

cmd = ['python', '-V'] if 'py' in sys.argv else ['pwd']
if 'shell' in sys.argv:
    subprocess.call(' '.join(cmd), shell=True)
else:
    subprocess.call(cmd)


td@timsworld2:~/tmp/so$ time ./nothing.py
/home/td/tmp/so

real    0m0.024s
user    0m0.012s
sys     0m0.008s
td@timsworld2:~/tmp/so$ time python nothing.py
/home/td/tmp/so

real    0m0.020s
user    0m0.012s
sys     0m0.004s
td@timsworld2:~/tmp/so$ time ./nothing.py py
Python 2.7.3

real    0m0.022s
user    0m0.016s
sys     0m0.000s
td@timsworld2:~/tmp/so$ time ./nothing.py sh
/home/td/tmp/so

real    0m0.020s
user    0m0.012s
sys     0m0.004s
td@timsworld2:~/tmp/so$