使用多处理在python3.7中并行运行不同的function

Question

I want to run two different function in parallel in python, I have used the below code:

def remove_special_char(data):
    data['Description'] = data['Description'].apply(lambda val: re.sub(r'^=', "'=", str(val))) # Change cell values which start with '=' sign leading to Excel formula issues
    return(data)

file_path1 = '.\file1.xlsx'
file_path2 = '.\file2.xlsx'


def method1(file_path1):
    data = pd.read_excel(file_path1)
    data= remove_special_char(data)
    return data

def method2(file_path2):
    data = pd.read_excel(file_path2)
    data= remove_special_char(data)
    return data

I am using the below Pool process, but its not working.

from multiprocessing import Pool

p = Pool(3)
result1 = p.map(method1(file_path1), args=file_path1) 
result2 = p.map(method2(file_path1), args=file_path2) 

I want to run both these methods in parallel to save execution time and at the same time get the return value as well.

Answer 1

I don't know why you are defining the same method twice with different parameter names, but anyway the map method of Pool s is taking as its first argument a function, and the second argument is an iterable. What map does is call the function on each item of the iterable, and return a list with all the results. So what you want to do is more something like:

from multiprocessing import Pool

file_paths = ('.\file1.xlsx', '.\file2.xlsx')

def method(file_path):
    data = pd.read_excel(file_path)
    data= remove_special_char(data)
    return data

with Pool(3) as p:
    result = p.map(method, file_paths)