Joblib是否需要多个内核进行多处理？

Question

I am using joblib for processing 4 frames (images taken from video) in parallel. Used the parallel for loop for this where each loop called the same function for different frames. Since the capturing of the frame takes just 1 percent of the total processing time (found by python script profiling using line profiler), I first capture 4 frames and hand over to joblib for multiprocessing.

I found that using Laptop, the process speeded up by roughly 5x. On Rapberry pi-2 (quad-core), almost 8x speedup was observed and on the Beaglebone Black, (not multicore) also, there was almost 7x speedup is observed. So, how exactly does joblib distribute the tasks?

Answer 1

By default joblib.parallel uses the Python multiprocessing module to fork separate Python worker processes to execute tasks concurrently on separate CPUs.

The multiprocessing module can take advantage of multiple cores/processors (it gets around GIL by starting multiple processes transparently). It offers some primitives similar to the threading module.

The overall efficiency depends on subtleties of how the cores share access to memory.