将多个 arguments（数据除外）传递给 imap function 用于 Python 中的多处理

Question

I am working on a simple multiprocessing program in python where I am passing a dataset where each pair of the list will return the multiplication of them. But when I am trying to pass a string along with the dataset to specify the operation (basically if I pass 'add' it will add the pair of numbers and if I pass 'sub' then it will subtract the pair of numbers) then I am facing all kinds of errors. I tried to use the concept of partial function also but it was not that helpful. Here I am attaching my basic code if anybody can help me to modify the code then it will be a great help

import multiprocessing as mp

data=([1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],
[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6]
,[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6],[1,2],[2,3],[3,4],[4,5],[5,6])

def mul(x,y):
   return(x*y)

def data_param(data):
    return mul(data[0],data[1])
def pool_handler():
    #p1=input('enter plus/mul: ')
    p=mp.Pool(3)
    
    out=p.imap(data_param,data)
    for i in out:
        print(f"output: {i}")
if __name__ == '__main__':
    pool_handler()

Thank you in advance

Answer 1

While not a direct solution to using imap, for multiprocessing I find apply_async more flexible in general use.

For the job here:

def mul(x,y):
    return(x*y)

def div(x,y):
    return(x/y)

def data_param(data, op):
    if op == "mul":
        return mul(data[0],data[1])
    elif op == "div":
        return div(data[0],data[1])
    else:
        print("You'll not get this print return in windows warning you of an error so beware!")


# ------------------------------------------------------------------
if __name___ == "__main__":

    data = [[1,2],[2,3],[3,4],[4,5],[5,6]]  #Cut down to reduce clutter

    op = "mul"

    pool = mp.Pool(mp.cpu_count())
    jobs = []

    for dataBit in data:
        jobs.append(pool.apply_async(data_param, args=(dataBit, op)))

    results = [] 
    for jobBit in jobs:
        results = jobBit.get()

    pool.close()

Of course, you could probably directly use the operator passed into data_param with some clever footwork - but for the purposes of verbosity here, best left simple!

Answer 2

A simple way is to pass the function along with its parameters:

...
def mul(x, y):
    return (x * y)

def add(x, y):
    return x + y

def data_param(data):
    return data[2](data[0], data[1])

def pool_handler():

    p1 = input('enter plus/mul: ')
    if p1 == 'plus':
        func = add
    elif p1 == 'mul':
        func = mul
    else:
        print('Incorrect value')
        exit(1)

    p = mp.Pool(3)

    out = p.imap(data_param, ((elt[0], elt[1], func) for elt in data))
    for i in out:
        print(f"output: {i}")