Python多处理从模块调用函数

Question

I have a my_module module with a my_func() function. When I use this function in the parent process, then everything works well (I get the correct result from this function). But when I call this function from child processes then I get this error:

AttributeError: module 'my_module' has no attribute 'my_func'

Does anybody know what I did wrong? Here is my template code:

import pandas as pd, gc, sys, multiprocessing as mp, traceback as tb
sys.path.append('c:/my_libs/')
import my_module

def mp_func(df, i):
    df['res'] = df.apply(lambda x: my_module.my_func(x, i))
    return df

def other_function(df):
    # do something
    my_list = [1,2,3] # it can change dynamically based on previous code

    df_res = pd.DataFrame()

    mp_pool = mp.Pool(processes=min(len(my_list), mp.cpu_count()))
    try:
        for i_df in [y.get() for y in [mp_pool.apply_async(mp_func\
            , [df[['my_attr']], i]) for i in my_list]]:

            df_res = df_res.append(i_df)

    except Exception as e:
        print('Something went wrong during multiprocessing:\n', e)
        print('\nFull traceback:\n', tb.format_exc())

    finally:
        mp_pool.close()
        gc.collect()

    df_res.to_csv(<...>)    

if __name__ == '__main__':
    df = pd.read_csv(<...>)
    other_function(df)

And my_module module looks like this:

def my_func(x, i):
    return x * i

Again, if I run the NOT multiprocessed version of the same code then everything works well. (I use python 3)

Thank you!

Answer 1

Finally I found the problem. I use Spyder, and in it the default working directory was set to an other folder where I had a previous version of my module. This topic helped me to print out the path of my imported modules in the child processes.