Python Pyro4远程对象块（如已死锁）

Question

I have a problem with Python Pyro4 remote objects which behave as if they were deadlock. Here's how to reproduce the problem (in Windows). Start the name server:

set PYRO_HMAC_KEY=some_key
python -m Pyro4.naming

Then run the remote object server:

import Pyro4
import sys


class Scheduler:
    def test(self):
        pass

if __name__ == '__main__': 

    sys.excepthook = Pyro4.util.excepthook

    scheduler = Scheduler()

    Pyro4.config.HMAC_KEY='some_key'  

    deamon = Pyro4.Daemon()
    ns = Pyro4.locateNS()
    ns.register("scheduler", deamon.register(scheduler))

    deamon.requestLoop()

Then run the client:

import sys
from multiprocessing import Process
import Pyro4

class BWModule(Process):

    def __init__(self):
        Process.__init__(self)
        self.depth = 1


    def run(self): 

        Pyro4.config.HMAC_KEY='some_key'       
        self.scheduler = Pyro4.Proxy("PYRONAME:scheduler")                

        print "1"
        sys.stdout.flush()         
        self.scheduler.test()
        print "2"
        sys.stdout.flush()         

        print "depth", self.depth
        sys.stdout.flush() 

        if self.depth < 5:
            for i in range(10):
                newblock = self.duplicate()
                newblock.depth = self.depth + 1
                newblock.start()


    def duplicate(self):
        dup = type(self)()
        return dup



if __name__ == '__main__': 

    sys.excepthook = Pyro4.util.excepthook

    No1 = BWModule()    
    No1.start()

When I run this code, I see printed message "depth X" with X from 1 to 3 but no more. I also see at the end of the execution (before nothing else happen) a bunch of 1s without corresponding 2s, which indicates that the call the the Pyro remote object "self.scheduler.test()" has blocked. This makes me think that this is a problem with Pyro rather than with processes (for example, running out of available processes if such a thing is possible). However, if I reduce the process multiplication from 10 to 2 (ie, replacing "for i in range(10):" by "for i in range(2):" in the client code), the execution goes all the way to depth 5 without blocking...

So here are my questions: What is happining? Why does it block with "for i in range(10):"? Is there a limit of possible "client" processes to a Pyro4 remote object or something like that? Is it a deadlocking problem?

Thank you.

Answer 1

Ok. For anyone facing a similar problem, this problem seems to be associated with some limit to the number of Pyro proxies that can hold simultaneously a reference on a remote object. Beware however that I'm only conjecturing this. This conclusion is based on the fact that adding

del self.scheduler

right before

if self.depth < 5:

in the client code both limits the number of alive proxies and fixes the problem (giving it enough time, the code go up to depth 5 using "for i in range(10):").

EDIT:

As I understand it from here and from some testing, the value of THREADPOOL_MINTHREADS is linked to the maximal number of connections or proxies to a remote object. If I set it to 4, the program freeze (much like if it was deadlock) when the fifth proxy tries to acces my remote object.

There is also some possibly related information here .

Answer 2

Just faced a similar issue, a neater way to do this might be to change

 self.scheduler = Pyro4.Proxy("PYRONAME:scheduler") 

to

 with Pyro4.Proxy("PYRONAME:scheduler") as self.scheduler:
    # Rest of code using self.scheduler
 #Rest of code not using self.scheduler

This is equivalent to using del self.scheduler except you don't need to worry about where you place this - python does the heavy lifting for you.