pool.apply_async, pool.map 上相当复杂的 function

Question

I have a function that computes an approximation of KL divergence in higher dimensions. Inside, it uses sklearn.NearestNeighbors and calculations on np.arrays. Its called KLnn(P) where P.shape = (10000,2). I want to apply it on many P inputs. [P0,P1,...]. So I applied to test it on one input:

if __name__ == '__main__':
    with Pool() as p:
        res = p.apply_async(KLnn, (P,))
        print(res.get(timeout = 1))       

I get a timeout, and it's not causes by the calculation of the function, as the function call is far below 1 sec. So I tried:

if __name__ == '__main__':
    with Pool() as p:
        print(p.map(KLnn, [P]))
        p.close()
        p.join()

which then starts a deadlock. In both cases, I would have expected that it just computes KLnn(P).

Both code chunks were just grabbed from the multiprocessing docs.python example.

Answer 1

I found out that OS Windows 10 together with spyder is not suited for the multiprocessing package. This toy example and apply_async in a list comprehension works well in the cmd / anaconda prompt. However, being new in the field of parallel processing it was not the first guess that the IDE / OS is the problem.