Need of running the same python file in parallel

Question

I have a python program running. It is basically does an ssh connection and sustains that connection ie the program does not enter bash shell again until terminated manually. Something like below,

bash-4.1$ python monitor.py
CONNECTION MADE
one done...
two done...
three done...
.
.
.

I want to same monitor.py to run in parallel for various such ssh connections. All of them are independent of each other and needn't exchange information. Can I achieve this using by Multithreading or Multiprocessing ?

Answer 1

You can make multiple threads to achieve what you have mentioned in your question. Below code is an example of multithreading in Python. Try it with your file.

import thread
import time

# Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s" % ( threadName, time.ctime(time.time()) )

# Create two threads as follows
try:
   thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print "Error: unable to start thread"

while 1:
   pass

Output

Thread-1: Thu Jan 22 15:42:17 2009
Thread-1: Thu Jan 22 15:42:19 2009
Thread-2: Thu Jan 22 15:42:19 2009
Thread-1: Thu Jan 22 15:42:21 2009
Thread-2: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:23 2009
Thread-1: Thu Jan 22 15:42:25 2009
Thread-2: Thu Jan 22 15:42:27 2009
Thread-2: Thu Jan 22 15:42:31 2009
Thread-2: Thu Jan 22 15:42:35 2009

Answer 2

Here is a similar example, that uses multiprocessing instead of multithreading (for documentation, see the official docs ). Mulitprocessing works very similarly to multithreading, but it circumvents the Global Interpreter Lock, therefore allowing your script to actually run different processes at the same time, and potentially making better use of limited computing resources.

import multiprocessing as mp

def my_function(*args):
    print("Arguments: {0}".format(args))


class MyProcess(mp.Process):
    def __init__(self, target, args):
        mp.Process.__init__(self, target=target, args=args)

def main():
    a1 = MyProcess(target=my_function, args=("1st Process...",))
    a2 = MyProcess(target=my_function, args=("2nd Process...",))
    a3 = MyProcess(target=my_function, args=("3rd Process...",))
    a4 = MyProcess(target=my_function, args=("4th Process...",))

    proclist = [a1, a2, a3, a4]
    for proc in proclist:
        proc.start()

    for proc in proclist:
        proc.join()

if __name__ == '__main__':
    main()

Output:

Arguments: ('1st Process...',)
Arguments: ('2nd Process...',)
Arguments: ('3rd Process...',)
Arguments: ('4th Process...',)

While these came in in what appears to be a set order, if you add a task that takes a non-deterministic time, they will come in in the order they finish. Just replace the contents of my_function with your code, and you should be set. (Note: this uses Python 2. The same works in Python 3 with very little modification--maybe none--but if you're in Python 3, you should also investigate concurrent.futures ).