Run multiple subprocesses in parallel - python 2.7

Question

The script shown below is my attempt at pinging multiple network namespaces using python 2.7 running on Linux (Fedora) OS.

Current status and problem :

When I run this file; the ping from elem1 (namespace) gets stored in a file called results.txt. But, I can't seem to get the loop to come back around and ping elem2, elem3, ... elemN

Attempted Fixes :

I tried killing the process using "kill -9 p.pid" (as shown) in the hope that this would kill the process and then a new process could be created on next iteration of the loop. However this is not the case! I've checked the documentation ( https://docs.python.org/2/library/subprocess.html ) and tried several different permutations of kill(), terminate(), removing shell=True etc... but to no avail.

import time
import os
import signal
import subprocess    

IP_ADDR="192.168.1.1"    

def main():
arry =["elem1", "elem2", "elem3", "elem4", "elem5", "elem6", "elem7"]  array of network namespaces's to ping

        with open('results.txt', 'a+') as outfile:
            for elem in arry:
                command = "ip netns exec {0} ping {1}".format(elem, IP_ADDR)
                outfile.write("\n\nPinging {}\n".format(elem))
                p = subprocess.Popen(command, shell=True, stdout=outfile)
                command_kill = "kill -9 {}".format(p.pid)
                time.sleep(2) #wait 5 seconds
                p.kill()
        outfile.close()

if __name__ == '__main__':
    main()

My Questions :

(1) Can someone explain what the code is doing here?

(2) Can you suggest a means to achieve my aforementioned goal?

Thank you

Answer 1

Along with the modifications mentioned by @chepner, you can try to use subprocess.call() instead of subprocess.Popen() . The latter method is not blocking and this causes all the commands to be executed simultaneously. However, call() is blocking and therefore your script will wait until the pinging is finished before entering next iteration of the loop. This will cause the output of your commands to be in sequential order and not interleaving.

If you need to execute the commands in parallel, I would suggest to write the outputs into different files and combine them after all commands are finished.

Edit: I have no particular experience in this area, but I guess the termination issue is related to the ping command only. Check the manual page here: https://linux.die.net/man/8/ping . In our case, we need to ping the destination X times. This is specified by using parameter -c X , where X defines the number of packets to be sent. This parameter can be also combined with parameter -w / -W which specify the timeout limit for the ping command. Take a look at some examples: https://www.thegeekstuff.com/2009/11/ping-tutorial-13-effective-ping-command-examples/