python 多处理池的保存数据在 linux 上没有给出预期结果

Question

I'm trying to run many calculations at the same time using multiprocessing pool in python. Each time a calculation is done I want to save the result in a file for later use. In the code below I'm trying to test this idea. On windows I get the expected result of different output for each calculation. The same code on Linux has a problem because it outputs the same result for all calculations. I don't understand how to solve this problem on Linux. Also, I would be happy to know how to save the data each time a calculations is done instead of waiting for all the calculations to be done and then saving all outputs at the same time. I've tried implementing different types of parallel computing with python using the examples on this site but couldn't get this issue fixed on Linux. I've also searched for the past two days for solutions and couldn't find or understand how to fix this. Any help would be really appreciated.

This is my code:

import multiprocessing
from functools import partial
import numpy as np
rng=np.random.default_rng()
import time

def sim(data_in_1, data_in_2, data_in_3, batch_num, run_number):

    file_str='run_'+str(run_number)+'.npy'
    temp=rng.choice(1000,5);
    time.sleep(temp[0]/1000)
    
    for i in range(temp[0]):
        a=np.sqrt(temp[1])
        
    return run_number, a, temp

def main():
    processes_num=8
    
    batch_num=1;
    num_of_calculations=8*2
    iterable=range(num_of_calculations);
    data_in_1=20;
    data_in_2=10;
    data_in_3=1;
    pool = multiprocessing.Pool(processes=processes_num)
    func = partial(sim, data_in_1, data_in_2, data_in_3, batch_num)
    results=pool.map(func, iterable)
    for r in results:
        out1=r
        file_str='run_'+str(out1[0])+'.npy'
        with open(file_str, 'wb') as f:
            np.save(f,out1[1])
            np.save(f,out1[2])
        print('saved run '+ str(out1[0]))
    pool.close()
    pool.join()
    
    print('Batch no. '+str(batch_num)+' is finished.')
    
    for run_number in range(num_of_calculations):
        file_str='run_'+str(run_number)+'.npy'
        with open(file_str, 'rb') as f:
            temp=np.load(f)
            temp=np.load(f)
        print('result of run ' + str(run_number) +' is: ' + str(temp))

if __name__ == "__main__":
    main()

Output on windows:

saved run 0
saved run 1
saved run 2
saved run 3
saved run 4
saved run 5
saved run 6
saved run 7
saved run 8
saved run 9
saved run 10
saved run 11
saved run 12
saved run 13
saved run 14
saved run 15
Batch no. 1 is finished.
result of run 0 is: [173 600 438 195 877]
result of run 1 is: [925 710 727 604 759]
result of run 2 is: [883 645 558 875 205]
result of run 3 is: [843 541 597 605 513]
result of run 4 is: [342 439 406 101 192]
result of run 5 is: [472 279 796  99 774]
result of run 6 is: [443 982  49 314 854]
result of run 7 is: [383  45 923 356 156]
result of run 8 is: [344 597 675 615 297]
result of run 9 is: [605 241 523 241 570]
result of run 10 is: [330 457 172 670 130]
result of run 11 is: [ 38 926 902 659 782]
result of run 12 is: [573 150 435 216 765]
result of run 13 is: [178 851 878 155 431]
result of run 14 is: [929 749 730 368 504]
result of run 15 is: [235 310 836 940 701]

Output of same code on Linux:

saved run 0
saved run 1
saved run 2
saved run 3
saved run 4
saved run 5
saved run 6
saved run 7
saved run 8
saved run 9
saved run 10
saved run 11
saved run 12
saved run 13
saved run 14
saved run 15
Batch no. 1 is finished.
result of run 0 is: [  9 218 388 265 856]
result of run 1 is: [  9 218 388 265 856]
result of run 2 is: [  9 218 388 265 856]
result of run 3 is: [  9 218 388 265 856]
result of run 4 is: [  9 218 388 265 856]
result of run 5 is: [  9 218 388 265 856]
result of run 6 is: [  9 218 388 265 856]
result of run 7 is: [  9 218 388 265 856]
result of run 8 is: [715 532 364 775 437]
result of run 9 is: [715 532 364 775 437]
result of run 10 is: [715 532 364 775 437]
result of run 11 is: [715 532 364 775 437]
result of run 12 is: [715 532 364 775 437]
result of run 13 is: [715 532 364 775 437]
result of run 14 is: [715 532 364 775 437]
result of run 15 is: [715 532 364 775 437]

Answer 1

So it turns out that the random number generator seed gets inherited from the parent in Linux, but on windows it is changed for every process. That's why I kept getting the same result on Linux. So I moved the line:

rng=np.random.default_rng()

from global to inside the function "sim" and that did the trick.