I have a mac (Mac Os 11.1, Python Ver 3.8.2) and need to work in multiprocessing, but the procedures doesn't work.
import multiprocessing
def func(index: int):
print(index)
manager = multiprocessing.Manager()
processes = []
for i in range(-1, 10):
p = multiprocessing.Process(target=func,
args=(i,))
processes.append(p)
p.start()
for process in processes:
process.join()
However, on my Intel-based Mac, it works fine.
What I expect is
-1
0
1
2
3
4
5
6
7
8
9
But instead, I got an error:
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
Traceback (most recent call last):
File "/Users/lance/Documents/Carleton Master Projects/Carleton-Master-Thesis/experiment.py", line 7, in <module>
manager = multiprocessing.Manager()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 57, in Manager
m.start()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/managers.py", line 583, in start
self._address = reader.recv()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 250, in recv
buf = self._recv_bytes()
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 414, in _recv_bytes
buf = self._recv(4)
File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/connection.py", line 383, in _recv
raise EOFError
EOFError
Is there any similar way (also keep it easy) to parallelize in M1-based Mac?
I'm not sure why this works on an Intel machine, but the problem is all the MP-related code should be inside if __name__ == '__main__':
import multiprocessing
def func(index: int):
print(index)
if __name__ == '__main__':
manager = multiprocessing.Manager()
processes = []
for i in range(-1, 10):
p = multiprocessing.Process(target=func, args=(i,))
processes.append(p)
p.start()
for process in processes:
process.join()
Since MP actually starts a new process, the new process needs to import the func
function from this module to execute its task. If you don't protect the code that starts new processes, it will be executed during this import.
The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.