py4J Error while obtaining a new communication channel on multithreaded java

Question

I'm having a problem when using py4J with a multithread Java application.

I have implemented a Java interface with Python

class PythonListener(object):
    def __init__(self):
        pass
    def onNewMessage(self, message):
        print(message)
    class Java:
        implements = ["com.myMessaging.IListener"]

I start the JVM from Python:

java_gateway = JavaGateway.launch_gateway(
                die_on_exit = True)

java_import(java_gateway.jvm, 'com.myMessaging.*')

// Start the multithread messaging server
myMessaging.Server.start()
// Register the listener
myMessaging.Server.registerListener(PythonListener())

myMessaging is a Java multithread application. When a message arrives, some thread will receive it and will call the onNewMessage method of the registered Listeners

In fact, when a message arrives i get this error

py4j.Py4JException: Error while obtaining a new communication channel
    at py4j.CallbackClient.getConnectionLock(CallbackClient.java:257)
    at py4j.CallbackClient.sendCommand(CallbackClient.java:377)
    at py4j.CallbackClient.sendCommand(CallbackClient.java:356)
    at py4j.reflection.PythonProxyHandler.invoke(PythonProxyHandler.java:106)
    at com.sun.proxy.$Proxy79.onNewMessage(Unknown Source)
    at com.myMessaging.Server.notifyMessage(Server.java:235)
       ...Some stuff...
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
    at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
    at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
    at java.net.Socket.connect(Socket.java:529)
    at java.net.Socket.connect(Socket.java:478)
    at java.net.Socket.<init>(Socket.java:375)
    at java.net.Socket.<init>(Socket.java:218)
    at javax.net.DefaultSocketFactory.createSocket(SocketFactory.java:212)
    at py4j.CallbackConnection.start(CallbackConnection.java:226)
    at py4j.CallbackClient.getConnection(CallbackClient.java:238)
    at py4j.CallbackClient.getConnectionLock(CallbackClient.java:250)
    ... 32 more

I think it is because there are multiple Java threads trying to connect to the same Python instance, on the same port, but how to fix it? I assumed that the system is multi threaded as described here in the Py4J guide .

Could it be an error in the implementation of the interface instead? Thank you for your helping!

Answer 1

When you use launch_gateway, the callback server that is responsible for executing Python callbacks is not started. You must explicitly start it:

java_gateway = JavaGateway.launch_gateway(die_on_exit = True)
java_gateway.start_callback_server()
# Your code here

Note that this will use the default port for Python callbacks, but you are using an ephemeral port for Java calls. If you want to use ephemeral ports for both Java and Python, the solution presented in the advanced topics section of Py4J is a bit more convoluted .