简体   繁体   中英

I do want each multiprocessing pool worker to have its own copy of the global variable and be able to modify it

I want each process to have it's own copy of global variable and process should be able to modify and store the global variable:

import multiprocessing
from multiprocessing import Pool
import time
input = [1,10]
first = 0
def printer (input) :
    global first
    first += input
    print('input', input)
    print('for input',input ,'first' , first)

def pool_handler():
    p = Pool(2)
    p.map(printer, input)


if __name__ == '__main__':
    while True:
        pool_handler()
        time.sleep(5)

My current output is

input 1
for input 1 first 1
input 10
for input 10 first 10 
input 1
for input 1 first 1
input 10
for input 10 first 10
...

WHere as my expected output is

input 1
for input 1 first 1
input 10
for input 10 first 10
input 1
for input 1 first 2
input 10
for input 10 first 20    

If we print process id, we can see each worker has its own variables:

import multiprocessing
from multiprocessing import Pool
import os
import time
input = [1,10]
first = 0
def printer (input) :
    global first
    first += input
    print(f'Process {os.getpid()} for input={input} first={first}')

def pool_handler(p):
    p.map(printer, input)

if __name__ == '__main__':
    p = Pool(2)
    while True:
        pool_handler(p)
        time.sleep(5)

I put pool allocation before the while loop.

So It is not possible with multiprocessing.pool to create different processes where each process has it's own copy of global variable. Since when pools are created all tasks are queued and and tasks are assigned to whichever pool is free.

So for my problem now i am using multiprocessing.process, I created two processes and now each has it's own copy of global variable, and each process can store and modify global variable. Below i am posting my current code.

import multiprocessing
from multiprocessing import Pool,process,Process
import os
import time
#input = [1,10]
first = 0
def printer (input) :
    while True :
        global first
        first += input
        print(f'Process {os.getpid()} for input={input} first={first}')
        time.sleep(2)

if __name__ == '__main__':
    p1 = multiprocessing.Process(target=printer, args=(1,))
    p2 = multiprocessing.Process(target=printer, args=(10,))
        p1.start()
        p2.start()

and I had to move while True code in my function because once a process is terminated it can not restart the same process.

The technical post webpages of this site follow the CC BY-SA 4.0 protocol. If you need to reprint, please indicate the site URL or the original address.Any question please contact:yoyou2525@163.com.

 
粤ICP备18138465号  © 2020-2024 STACKOOM.COM