Python Multiprocessing Pool With Infinite While Loop

Question

I am attempting to run an infinite while loop that will call a function that makes an API call. Each API call can take between 9-12 seconds and I want x processes constantly running.

I've attempted some of the existing pool examples but they seem to take a function along with a list of arguments. My function generates all inputs needed.

from multiprocessing import Process
from random import randint
from time import sleep

def loop_a():
    while 1:
        wait = randint(5,9)
        print("now waiting" + str(wait) + " seconds")
        sleep(wait)

if __name__ == '__main__':
    #Initialize two separate while loops that can call/wait for responses independently  
    Process(target=loop_a).start()
    Process(target=loop_a).start()

This sample code I found from another question solves my problem mostly, but I am wondering if there is an elegant way to define how many processes to run at once. Id like to be able to enter the number of processes as a parameter rather than defining a new line for each process. Is there a nice way to accomplish this?

This snippet seems to fix the issue I was facing.

if __name__ == '__main__':
    [Process(target=loop_a).start() for x in range(2)]

Answer 1

You should use Pool

from multiprocessing import Pool
pool = Pool(processes=2) # define number of processes
results = [pool.apply(loop_a) for x in range(2)]