I have a function returning a tuple of two elements. The function is called with pool starmap to generate a list of tuples which are unpacked to two lists.
def func():
#...some operations
return (x,y)
def MP_a_func(func,iterable,proc,chunk):
pool=multiprocessing.Pool(processes=proc)
Result=pool.starmap(func,iterable,chunksize=chunk)
pool.close()
return Result
##
if __name__ == '__main__':
results=MP_a_func(func,iterable,proc,chunk)
a,b=zip(*results)
I now wish to use dask delayed
API as the following
if __name__ == '__main__':
results=delayed(MP_a_func(func,iterable,proc,chunk))
is it possible to unpack tuples in the delayed object without using results.compute()
?
Thank your for your help
It is possible for another delayed function to unpack the tuple, in the example below, the delayed value of return_tuple(1)
was not computed, but passed as a delayed
object:
import dask
@dask.delayed
def return_tuple(x):
return x+1, x-1
@dask.delayed
def process_first_item(some_tuple):
return some_tuple[0]+10
result = process_first_item(return_tuple(1))
dask.compute(result)
As per @mdurant's answer, it turns out delayed
function/decorator has nout
parameter, also see this answer .
If you know the number of outputs, the delayed
function (or decorator) takes an optional nout
arguments, and this will split the single delayed into that many delayed outputs. This sounds like exactly what you need.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.