Python Twisted reactor shared across python processes?

Question

Trying to ask a basic yes / no question - to which the answer I fear is "no" - but if it is a "yes", seeking guidance on how!

I have an SMTP server written in Twisted Python. It works very nicely! Basic flow / sequence of function calls is something like:

script1.py
from mailserver import startReactor, startListener, stopListener, stopReactor
startReactor()
startListener("p25")
startListener("p26")
# Handle incoming connections etc until sigint receivedand then exit cleanly
stopListener("p25")
stopListener("p26")
stopReactor()

What I would like to do, in one or more separate python scripts (well, actually it'll be RobotFramework tests, but the principal is the same) is:

script2.py
from mailserver import startReactor, startListener, stopListener, stopReactor
startListener("p27")
# Handle incoming connections etc until all necessary stuff on p27 is complete
stopListener("p27")

And finally

script3.py
from mailserver import startReactor, startListener, stopListener, stopReactor
stopListener("p25")
startListener("p25custom)
# Handle incoming connections etc until all necessary stuff on "custom" p27 is complete
stopListener("p25custom")
startListener("p25)

So the idea is that script1.py is executed in the background,and then script2, script3 etc can be executed, "changing" the list of running listeners, but with these listeners attached to the reactor in script1....

By monitoring ps -xaf and netstat I can see the sockets opening in all scripts, and script1 exits cleanly ... but the sockets opened in script2 and script3 don't seem to close down...

In mailserver.py I maintain a dict of "runningListeners" (eg: {'p25': <<class 'twisted.internet.tcp.Port'> of mailserver.ConsoleSMTPFactory on 25>} ) which gets added to / deleted from as startListener and stopListener get called as appropriate. However, this is obviously local to just script1, and not a "shared" dict between the 3 scripts... And I very much doubt the listeners started in script2 and script3 are actually as "attached" to the reactor started in script1 as netstat / ps might suggest - and as such probably not "useable" listeners...

So the yes no question - is it even possible to do what I am trying to do with multiple python scripts, and if so, can anyone offer a suggestion on how I can achieve this?

Many Thanks!

Answer 1

One solution would be to make script1 'smarter' so that it runs the one single twisted reactor, but also have it running additional polling code that monitors either the file system or the network for bespoke 'instructions' sent from the other two scripts.

The instructions from the other two scripts would then instruct script1 about what to listen for (eg what ports etc to listen on and what handler code it should to run when an event occurs, possibly delegating using the subprocess module) and also indicate when a listener should be stopped.

So I guess the answer is erm yes, but its not easy so you may wish to reconsider your design first.