JMX different port forward from kubernetes pod

Question

I've got problem forwarding jmx from kubernetes pod to localhost. Everything works fine when port is forwarded to the same port:

kubectl port-forward ... 9010:9010 OR
kubectl port-forward ... 9010

However, when I try to forward to a different local port,

kubectl port-forward ... 9011:9010

neither jconsole or jms can connect.

It is blocking me from profiling multiple applications at once.

JMS Error:

com.oracle.jmc.rjmx.ConnectionException caused by java.rmi.NoSuchObjectException: no such object in table
    at com.oracle.jmc.rjmx.internal.RJMXConnection.connect(RJMXConnection.java:406)
    at com.oracle.jmc.rjmx.internal.ServerHandle.doConnect(ServerHandle.java:88)
    at com.oracle.jmc.rjmx.internal.ServerHandle.connect(ServerHandle.java:78)
    at com.oracle.jmc.console.ui.editor.internal.ConsoleEditor$ConnectJob.run(ConsoleEditor.java:73)
    at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55)
Caused by: java.rmi.NoSuchObjectException: no such object in table
    at java.rmi/sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
    at java.rmi/sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
    at java.rmi/sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
    at java.management.rmi/javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
    at java.management.rmi/javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2105)
    at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:321)
    at com.oracle.jmc.rjmx.internal.RJMXConnection.connectJmxConnector(RJMXConnection.java:451)
    at com.oracle.jmc.rjmx.internal.RJMXConnection.establishConnection(RJMXConnection.java:427)
    at com.oracle.jmc.rjmx.internal.RJMXConnection.connect(RJMXConnection.java:399)
    ... 4 more

jconsole error:

Connection to localhost:9011 did no succeed.

Answer 1

This is an RMI issue. The problem is that the RMI stub delivered to the JMX client is created for 9010 and so it fails when attempting to connect at 9011. There's a couple of decent solutions outlined here . Another option is to switch to JMXMP which is a pure socket JMX protocol so port forwarding works without any additional workarounds.