使用多处理从两个不同的目录并行运行两个 python 代码

Question

Below is my code for running two python code in parallel using multiprocessing:

defs.py

import os 

def pro(process):      
    #print(process)                                                       
    os.system('python {}'.format(process)) 

Multiprocessing.py

import os                                                                       
from multiprocessing import Pool
import multiprocessing as mp
import defs
import datetime
import pandas as pd

processes = ('python_code1.py','python_code2.py')

if __name__ == '__main__':                                                                                
    pool = Pool(processes=4)
    start = datetime.datetime.now()
    print('Start:',start)   
    pool.map(defs.pro, processes)
    end = datetime.datetime.now()
    print('End :',end)   
    total = end-start
    print('Total :', end-start)

This code is running perfectly fine. But my requirement is I need to run the python code 'python_code1.py' and 'python_code2.py' from two different directory.

so I made the below changes in Multiprocessing.py :

path1 = r'C:\Users\code1\python_code1.py'
path2 = r'C:\Users\code2\python_code2.py'

processes = (path1,path2) 

but this is not working for me.

My Multiprocessing.py and defs.py are kept on path `C:\Users\Multiprocessing\'

Answer 1

Well an elegant solution using asyncio . It is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc. Plus it has both high-level and low-level APIs to accomodate any kind of problem. And you might find syntax easier as I do:

import os
import asyncio
    def background(f):
        def wrapped(*args, **kwargs):
            return asyncio.get_event_loop().run_in_executor(None, f, *args, **kwargs)

        return wrapped

    @background
    def pro(process):      
    #print(process)                                                       
    os.system('python {}'.format(process)) 

processes = (r'C:\Users\code1\python_code1.py',r'C:\Users\code2\python_code2.py')
for process in processes:
    pro(process)

Detailed answer on parallelizing for loop. You might find useful.