[英]Return multiprocessed outputs in python
I am trying to make two functions run in parallel.我试图让两个函数并行运行。 The first functions makes an API call and returns the json output and the second function makes a call to database and returns the data captured.
第一个函数进行 API 调用并返回 json output 和第二个 ZC1C425268E68385D1AB5074 调用数据库。
I have the following code -我有以下代码 -
import multiprocessing
ret = {'db': None, 'api':None}
def db_call(queue=None):
engine = db.create_engine('mysql+pymysql://{}:{}@{}/{}'.format(user, password, host, database))
dbConnection = engine.connect()
df_aws_accounts = pd.read_sql(query, dbConnection)
if queue:
queue['db'] = df_aws_accounts
return df_aws_accounts
def api_call(queue=None):
data = requests.get(url, verify=False)
df = pd.DataFrame(data)
if queue:
queue['api'] = df
return df
def runInParallel(*fns):
queue = multiprocessing.Queue()
queue.put(ret)
proc = []
for fn in fns:
p = Process(target=fn,args=((queue),))
p.start()
proc.append(p)
print(queue.get())
for p in proc:
p.join()
l = [api_call, db_call]
runInParallel(l)
when I run the above code -当我运行上面的代码时 -
Process Process-2:
Traceback (most recent call last):
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
self.run()
File "/usr/local/Cellar/python/3.7.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
TypeError: 'list' object is not callable
How do I get output from runInParallel and assign that to the variables?如何从 runInParallel 获取 output 并将其分配给变量? Edit- Note - These functions are working individually.
编辑-注意-这些功能是单独工作的。 but not when I do it via runInParallel function.
但不是当我通过 runInParallel function 执行此操作时。
Edit 2 - Updated the code based on suggestions.编辑 2 - 根据建议更新了代码。
The main problem is that, the function runInParallel
is not returning anything主要问题是,function
runInParallel
没有返回任何东西
But also if you want to make a function which is using multiprocessing return anything, you can create a global variable, like output, and make the function assign value to it like:但是,如果您想创建一个使用多处理返回任何内容的 function,您可以创建一个全局变量,如 output,并让 function 为它赋值:
def api_call():
global output
output = api_data
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.