如何在多处理中使用共享内存的概念

Question

I want to start 2 processes simultaneously using a shared variable. One will start processing immediately and the other will wait for the trigger (and the shared variable) from the first process , for it to start processing.

My first process, calculates the distance, and the 2nd process, acts differently based upon the distance traveled. Distance is passed as an argument and current_conveyer is the shared memory variable.

Following is my code:-

def process1():

    current_conveyer = Value('d', 'SC1')   # also I want to know how to initialize the string values. Current it is double precision float.

    while condition:
        conveyer_type = current_conveyer.value
        S = pickle.load(open('conveyer_speed.p','rb'))[conveyer_type]
        D = S * T # speed is changing, hence calculating the speed at every instant.
        # trigger the second process. NOT create a new process
        time.sleep(0.005)

def process2(current_converyer,distance):
    while True:
        if some condition:
              current_converyer = 'SC2'
        elif some condition:
              current_converyer = 'SC3'

As of right now, I'm starting a new process for every while loop.

I want to create a single process for all of this which will be listening and sharing the variable. If any trigger is sent, that process should listen, wake up, and work instead of creating a completely new process.

I've know this can be done via queues and pipes, but then using queues and pipes will defeat the purpose of shared memory.

I've tried implementing the above code alone with both queues and pipes, there were some time efficiency issues, hence now want to try out the shared memory variable method.

So given the above, I am wondering how do I keep the process listening and implement the shared memory concept as well.

Answer 1

The simplest way to do multiprocessing is having a Pool to execute several, larger jobs, each of which returns data to the parent process.

If you really want to have a 'shared' variable, create a Manager object, and use it to create managed (shared) objects. In this way multiple procs can read and write different values, and the values are communicated to the different children processes.

The following code has simple "main" code that starts two children then waits for them. One child wakes up, appends an item to a managed (shared) list, then waits. The second child waits for a while, consumes and prints the shared list, then sleeps.

source

import multiprocessing, signal, time

def producer(objlist):
    '''
    add an item to list every sec
    '''
    while True:
        try:
            time.sleep(1)
        except KeyboardInterrupt:
            return
        msg = 'ding: {:04d}'.format(int(time.time()) % 10000)
        objlist.append( msg )
        print msg


def scanner(objlist):
    '''
    every now and then, consume objlist & run calculation
    '''
    while True:
        try:
            time.sleep(3)
        except KeyboardInterrupt:
            return
        print 'items: {}'.format( list(objlist) )
        objlist[:] = []


def main():

    # create obj sharable between all processes
    manager = multiprocessing.Manager()
    my_objlist = manager.list() # pylint: disable=E1101

    multiprocessing.Process(
        target=producer, args=(my_objlist,),
    ).start()

    multiprocessing.Process(
        target=scanner, args=(my_objlist,),
    ).start()

    # kill everything after a few seconds
    signal.signal(
        signal.SIGALRM, 
        lambda _sig,_frame: manager.shutdown(),
        )
    signal.alarm(12)

    try:
        manager.join() # wait until both workers die
    except KeyboardInterrupt:
        pass


if __name__=='__main__':
    main()

output

ding: 8392
ding: 8393
ding: 8394
ding: 8395
ding: 8396
ding: 8397