[英]developing a script running the same file in parallel in python
I have a python script, it invokes another python file and runs it. 我有一个python脚本,它调用另一个python文件并运行它。 However I need the script to run the same file in parallel for multiple times.
但是,我需要该脚本多次并行运行同一文件。 I will share the code snippet here .This runs the python file once.
我将在这里共享代码片段。这将运行一次python文件。
output = os.popen('python py_generator_sm20.py' + options)
print output.read()
How do I parallelize it to run multiple times simultaneously? 如何使其并行运行多次?
This maybe not full answer you, because of que
output
part of your code, but could be a start point.
由于代码的
Using the output
部分,这可能无法完全回答您,但可能是一个起点。
multiprocessing
module you can create a pool of workers and then with the subprocess
module you can call one instance of your script for each worker and check the output : 使用
multiprocessing
模块,您可以创建一个工作池,然后使用subprocess
模块,可以为每个工作器调用脚本的一个实例,并检查输出 :
import multiprocessing as mp
import subprocess as sp
# an example with two runs
commands = ['python test.py', 'python test.py']
# pass the number of threads that will be working
# if the number of threads < len(commands) the exceed
# will run in sequence when some process terminate
pool = mp.Pool(processes=2)
# execute the script calls
res = pool.map(sp.check_output, commands)
print(*[item.decode() for item in res])
pool.close()
Attention: the return from check_output
is a byte string
, so you need to convert it back to string
注意:
check_output
的返回是一个byte string
,因此您需要将其转换回string
I tested it with the following simple program: 我使用以下简单程序对其进行了测试:
import time
if __name__ == "__main__":
print("Running an instance at {}".format(time.ctime()))
time.sleep(2)
print("Finished at {}".format(time.ctime()))
And that is the output: 这就是输出:
Running an instance at Thu Oct 11 23:21:44 2018
Finished at Thu Oct 11 23:21:46 2018
Running an instance at Thu Oct 11 23:21:44 2018
Finished at Thu Oct 11 23:21:46 2018
As you can see they runned at same time. 如您所见,它们同时运行。
I think you need this: 我认为您需要这样做:
from multiprocessing.dummy import Pool as ThreadPool
pt = ThreadPool(4)
results = pt.map(pt_function, pt_array)
or maybe this way (if you have many threading scripts): 或者也许是这种方式(如果您有很多线程脚本):
from Threading_orders import Thread
class First_time(Thread):
"""
A threading example
"""
def __init__(self, a, b):
"""Инициализация потока"""
Thread.__init__(self)
self.a = a
self.b = b
def run(self):
MyThread_1(self.a, self.b)
MyThread_2(self.a, self.b)
MyThread_3(self.a, self.b)
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.