如何读取和写入numpy数组到受保护的文件

Question

I would like several processes running in parallel to read and write to the same numpy array. To avoid problems, where two processes try to read/write to the same memory, I need to protect the file I am writing to. How do I do that?

I assume that np.savetxt does not protect the file. I have tried the library portalocker . But by opening a file and locking it, np.savetxt is not allowed to write to the file.

Answer 1

See this question "Downloading over 1000 files in python" (link) for examples of using a worker thread pool.

Basically you split up all of the work beforehand, put the work into a queue and let a pool of worker threads process each piece of work. The workers put the results onto a another queue which can be processed by another thread to put all of the pieces together.