python 3中的多处理和多线程

Question

I know that there is a GIL in python that forces threads to execute on only 1 core. But I created processes by the number of processor cores, and in each process I create threads. In theory, will they be executed in parallel in each process? And if it works, how can I synchronize everything while using Pool

from multiprocessing import Pool
from concurrent.futures import ThreadPoolExecutor

def make_threads(data):
 with ThreadPoolExecutor(len(data)) as executor:
     answer=list(executor.map(some_function,data))
     return answer

def main():
with Pool(processes_count) as p:
         answer=list(p.map(make_threads,data))```

Answer 1

Use concurrent.futures instead of multiprocessing or multithreading if you can - it's a better API that allows you to conveniently switch from threads to processes, or vice-versa.

Within a given python process, threads can do I/O bound tasks well, but CPU-bound tasks poorly. And if you have 20 I/O bound threads and 1 CPU-bound thread in a single process, then they all will have performance problems - the CPU-bound thread messes up the I/O bound threads.

Often Queues are the best way to communicate across processes and threads.

• Better: How would I go about using concurrent.futures and queues for a real-time scenario?
• Acceptable: https://docs.python.org/3/library/multiprocessing.html